Image processing apparatus and image processing method

ABSTRACT

There is provided an image processing apparatus and an image processing method capable of suppressing a reduction in encoding efficiency. The number of significant figures of a prediction residual of an image is expanded on the basis of a bit depth indicating a range of pixel values of a local level that is a data unit smaller than a sequence level of the image; the number of significant figures of a quantized coefficient obtained by performance of orthogonal transform and quantization on the prediction residual the number of significant figures of which is expanded is normalized on the basis of the bit depth of the local level; and the quantized coefficient the number of significant figures of which is normalized is encoded and a bit stream is generated. The present disclosure is applicable to, for example, an image processing apparatus, an image encoding apparatus, or an image decoding apparatus.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a National Stage Patent Application of PCTInternational Patent Application No. PCT/JP2019/003276 (filed on Jan.31, 2019) under 35 U.S.C. § 371, which claims priority to JapanesePatent Application No. 2018-024411 (filed on Feb. 14, 2018), which areall hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to an image processing apparatus and animage processing method and particularly relates to an image processingapparatus and an image processing method capable of suppressing areduction in encoding efficiency.

BACKGROUND ART

There has conventionally been known a method of controlling computingprecision of orthogonal transform and quantization (inverse quantizationand inverse orthogonal transform) on the basis of a bit depth ‘BitDepth’of an input image specified by a sequence parameter set (SPS) in imageencoding and decoding (refer to, for example, NPL 1).

For example, a scaling parameter ‘bdShift’ for controlling the computingprecision of the orthogonal transform and quantization (inversequantization and inverse orthogonal transform) is set on the basis ofthe bit depth ‘BitDepth’ set per sequence (specified by the SPS) asdescribed above, in the image encoding and decoding described in NPL 1.

CITATION LIST Non Patent Literature

-   [NPL 1]

J TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (InternationalTelecommunication Union), “High efficiency video coding”, H.265, 12/2016

SUMMARY Technical Problems

The bit depth of the input image changes in a time direction or aspatial direction within a sequence, for example, per picture or perlocal area within each picture. With the conventional method, however,it has been difficult to control the computing precision of theorthogonal transform and the quantization (the inverse quantization andthe inverse orthogonal transform) in the time direction or the spatialdirection within the sequence according to such a change in the bitdepth of the input image. Owing to this, there is a concern of areduction in encoding efficiency.

The present disclosure has been achieved in light of such circumstances,and an object of the present disclosure is to enable suppression of areduction in encoding efficiency.

Solution to Problems

An image processing apparatus according to one aspect of the presenttechnology is an image processing apparatus including an expansionsection that expands the number of significant figures of a predictionresidual of an image on the basis of a bit depth indicating a range ofpixel values of a local level that is a data unit smaller than asequence level of the image, a normalization section that normalizes thenumber of significant figures of a quantized coefficient obtained byperformance of orthogonal transform and quantization on the predictionresidual the number of significant figures of which is expanded by theexpansion section, on the basis of the bit depth of the local level, andan encoding section that encodes the quantized coefficient the number ofsignificant figures of which is normalized by the normalization sectionand that generates a bit stream.

An image processing method according to one aspect of the presenttechnology is an image processing method including expanding the numberof significant figures of a prediction residual of an image on the basisof a bit depth indicating a range of pixel values of a local level thatis a data unit smaller than a sequence level of the image, normalizingthe number of significant figures of a quantized coefficient obtained byperformance of orthogonal transform and quantization on the predictionresidual the number of significant figures of which is expanded, on thebasis of the bit depth of the local level, encoding the quantizedcoefficient the number of significant figures of which is normalized,and generating a bit stream.

An image processing apparatus according to another aspect of the presenttechnology is an image processing apparatus including an expansionsection that expands the number of significant figures of a predictionresidual of an image on the basis of a bit depth indicating a range ofpixel values of a local level that is a data unit smaller than asequence level of a predicted image corresponding to the image or of adecoded image referred to at a time of generation of the predictedimage, a normalization section that normalizes the number of significantfigures of a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level of the predicted image orthe decoded image, and an encoding section that encodes the quantizedcoefficient the number of significant figures of which is normalized bythe normalization section and that generates a bit stream.

An image processing method according to another aspect of the presenttechnology is an image processing method including expanding the numberof significant figures of a prediction residual of an image on the basisof a bit depth indicating a range of pixel values of a local level thatis a data unit smaller than a sequence level of a predicted imagecorresponding to the image or of a decoded image referred to at a timeof generation of the predicted image, normalizing the number ofsignificant figures of a quantized coefficient obtained by performanceof orthogonal transform and quantization on the prediction residual thenumber of significant figures of which is expanded, on the basis of thebit depth of the local level of the predicted image or the decodedimage, encoding the quantized coefficient the number of significantfigures of which is normalized, and generating a bit stream.

An image processing apparatus according to still another aspect of thepresent technology is an image processing apparatus including a decodingsection that decodes a bit stream, an expansion section that expands thenumber of significant figures of a quantized coefficient obtained bydecoding of the bit stream by the decoding section, on the basis of abit depth indicating a range of pixel values of a local level that is adata unit smaller than a sequence level, and a normalization sectionthat normalizes the number of significant figures of residual dataobtained by performance of inverse quantization and inverse orthogonaltransform on the quantized coefficient the number of significant figuresof which is expanded by the expansion section, on the basis of the bitdepth of the local level.

An image processing method according to still another aspect of thepresent technology is an image processing method including decoding abit stream, expanding the number of significant figures of a quantizedcoefficient obtained by decoding of the bit stream, on the basis of abit depth indicating a range of pixel values of a local level that is adata unit smaller than a sequence level, and normalizing the number ofsignificant figures of residual data obtained by performance of inversequantization and inverse orthogonal transform on the quantizedcoefficient the number of significant figures of which is expanded, onthe basis of the bit depth of the local level.

An image processing apparatus according to yet another aspect of thepresent technology is an image processing apparatus including a decodingsection that decodes a bit stream, an expansion section that expands thenumber of significant figures of a quantized coefficient obtained bydecoding of the bit stream by the decoding section, on the basis of abit depth indicating a range of pixel values of a local level that is adata unit smaller than a sequence level of a predicted image or of adecoded image referred to at a time of generation of the predictedimage, and a normalization section that normalizes the number ofsignificant figures of residual data obtained by performance of inversequantization and inverse orthogonal transform on the quantizedcoefficient the number of significant figures of which is expanded bythe expansion section, on the basis of the bit depth of the local levelof the predicted image or the decoded image.

An image processing method according to yet another aspect of thepresent technology is an image processing method including decoding abit stream, expanding the number of significant figures of a quantizedcoefficient obtained by decoding of the bit stream, on the basis of abit depth indicating a range of pixel values of a local level that is adata unit smaller than a sequence level of a predicted image or of adecoded image referred to at a time of generation of the predictedimage, and normalizing the number of significant figures of residualdata obtained by performance of inverse quantization and inverseorthogonal transform on the quantized coefficient the number ofsignificant figures of which is expanded, on the basis of the bit depthof the local level of the predicted image or the decoded image.

In the image processing apparatus and method according to one aspect ofthe present technology, the number of significant figures of aprediction residual of an image is expanded on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of the image, the number of significantfigures of a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded is normalized on the basis ofthe bit depth of the local level, the quantized coefficient the numberof significant figures of which is normalized is encoded, and a bitstream is generated.

In the image processing apparatus and method according to another aspectof the present technology, the number of significant figures of aprediction residual of an image is expanded on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of a predicted image corresponding to theimage or of a decoded image referred to at a time of generation of thepredicted image, the number of significant figures of a quantizedcoefficient obtained by performance of orthogonal transform andquantization on the prediction residual the number of significantfigures of which is expanded is normalized, on the basis of the bitdepth of the local level of the predicted image or the decoded image,the quantized coefficient the number of significant figures of which isnormalized is encoded, and a bit stream is generated.

In the image processing apparatus and method according to still anotheraspect of the present technology, a bit stream is decoded, the number ofsignificant figures of a quantized coefficient obtained by decoding ofthe bit stream is expanded on the basis of a bit depth indicating arange of pixel values of a local level that is a data unit smaller thana sequence level, and the number of significant figures of residual dataobtained by performance of inverse quantization and inverse orthogonaltransform on the quantized coefficient the number of significant figuresof which is expanded is normalized on the basis of the bit depth of thelocal level.

In the image processing apparatus and method according to yet anotheraspect of the present technology, a bit stream is decoded, the number ofsignificant figures of a quantized coefficient obtained by decoding ofthe bit stream is expanded on the basis of a bit depth indicating arange of pixel values of a local level that is a data unit smaller thana sequence level of a predicted image or of a decoded image referred toat a time of generation of the predicted image, and the number ofsignificant figures of residual data obtained by performance of inversequantization and inverse orthogonal transform on the quantizedcoefficient the number of significant figures of which is expanded isnormalized on the basis of the bit depth of the local level of thepredicted image or the decoded image.

Advantageous Effect of Invention

According to the present disclosure, it is possible to process an image.It is particularly possible to suppress a reduction in encodingefficiency.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram depicting an example of a state of control overcomputing precision.

FIG. 2 is a diagram depicting an example of a bit range to be used.

FIG. 3 is a diagram depicting an example of a state of a change incomputing precision.

FIG. 4 is a diagram depicting an example of a state of a change in bitdepth between pictures.

FIG. 5 is a diagram depicting an example of a state of a change in bitdepth of a luminance signal between local areas.

FIG. 6 is a diagram depicting an example of a state of a change in bitdepth of a luminance signal between local areas.

FIG. 7 is a diagram depicting an example of a state of a change in bitdepth of a chrominance signal between local areas.

FIG. 8 is a diagram depicting an example of a state of a change in bitdepth of a chrominance signal between local areas.

FIG. 9 is a diagram depicting a list of principal computing precisioncontrol methods.

FIG. 10 is a diagram depicting an example of a state of control overcomputing precision of orthogonal transform and quantization by method#1.

FIG. 11 is a diagram depicting an example of a state of control overcomputing precision of inverse quantization and inverse orthogonaltransform by the method #1.

FIG. 12 is a diagram depicting an example of a state of change incomputing precision.

FIG. 13 is a diagram depicting an example of a state of control overcomputing precision of orthogonal transform and quantization by method#2.

FIG. 14 is a diagram depicting an example of a state of control overcomputing precision of inverse quantization and inverse orthogonaltransform by the method #2.

FIG. 15 is a block diagram depicting an example of principalconfigurations of an image encoding apparatus.

FIG. 16 is a block diagram depicting an example of principalconfigurations of a control section.

FIG. 17 is a flowchart illustrating an example of a flow of imageencoding processing.

FIG. 18 is a flowchart illustrating an example of a flow of dBD anddeltaX derivation processing.

FIG. 19 is a diagram depicting an example of an input image.

FIG. 20 is a diagram depicting an example of an extension bit precisioncontrol group.

FIG. 21 is a diagram depicting an example of a method of deriving apredicted value of the extension bit precision.

FIG. 22 is a flowchart illustrating an example of a flow of extensionbit precision information encoding processing.

FIG. 23 is a diagram depicting an example of a syntax.

FIG. 24 is a block diagram depicting an example of principalconfigurations of an image decoding apparatus.

FIG. 25 is a block diagram depicting an example of principalconfigurations of a decoding section.

FIG. 26 is a flowchart illustrating an example of a flow of imagedecoding processing.

FIG. 27 is a flowchart illustrating an example of a flow of dBDderivation processing.

FIG. 28 is a block diagram depicting an example of principalconfigurations of an image encoding apparatus.

FIG. 29 is a block diagram depicting an example of principalconfigurations of a control section.

FIG. 30 is a flowchart illustrating an example of a flow of imageencoding processing.

FIG. 31 is a flowchart illustrating an example of a flow of sequence bitdepth setting processing.

FIG. 32 is a flowchart illustrating an example of a flow of dBDderivation processing.

FIG. 33 is a flowchart illustrating an example of a flow of extensionbit precision information encoding processing.

FIG. 34 is a block diagram depicting an example of principalconfigurations of an image decoding apparatus.

FIG. 35 is a block diagram depicting an example of principalconfigurations of a decoding section.

FIG. 36 is a flowchart illustrating an example of a flow of imagedecoding processing.

FIG. 37 is a flowchart illustrating an example of a flow of sequence bitdepth derivation processing.

FIG. 38 is a block diagram depicting an example of principalconfigurations of a computer.

DESCRIPTION OF EMBODIMENTS

Modes for carrying out the present disclosure (hereinafter, referred toas “embodiments”) will be described below. It is noted that thedescription will be given in the following order.

1. Control over computing precision

2. Common concept (outline of each approach)

3. First embodiment (details of method #1)

4. Second embodiment (details of method #2)

5. Notes

1. Control Over Computing Precision

<Documents and the Like Supporting Technical Contents and TechnicalTerms>

The scope disclosed in the present technology includes not only thecontents described in embodiments but also the contents described in thefollowing pieces of NPL that were well known at the time of filing ofthe present application.

-   NPL 1: (described above)-   NPL 2: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU    (International Telecommunication Union), “Advanced video coding for    generic audiovisual services”, H.264, 04/2017-   NPL 3: Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer,    Jill Boyce, “Algorithm Description of Joint Exploration Test Model    4”, JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG    16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT, 13-21    Jul. 2017

In other words, the contents described in the above pieces of NPL alsoform the basis for determining support requirements. It is assumed, forexample, that Quad-Tree Block Structure described in NPL 1 and QTBT(Quad Tree Plus Binary Tree) Block Structure described in NPL 3 arewithin the scope of disclosure of the present technology and satisfysupport requirements for claims even if direct description of suchstructures is not made in the embodiment. In addition, it is assumedthat technical terms such as parsing, syntax, and semantics aresimilarly within the scope of the disclosure of the present technologyand satisfy support requirements for claims even if direct descriptionof such terms is not made in the embodiment.

Furthermore, a “block” (not a block indicating a processing section)used as a partial area or a processing unit of an image (picture) indescription indicates any partial area within the picture, and amagnitude, a shape, characteristics, and the like thereof are notlimited to specific ones unless particularly mentioned otherwise. It isassumed that examples of the “block” include any partial areas(processing units) such as a TB (Transform Block), a TU (TransformUnit), a PB (Prediction Block), a PU (Prediction Unit), a SCU (SmallestCoding Unit), a CU (Coding Unit), an LCU (Largest Coding Unit), a CTB(Coding Tree Block), a CTU (Coding Tree Unit), a transform block, asub-block, a macroblock, a tile, and a slice described in NPL 1 to NPL3.

Moreover, in designating a size of such a block, a block size may bedesignated not only directly but also indirectly. For example, the blocksize may be designated using identification information for identifyingthe size. Alternatively, the block size may be designated by, forexample, a ratio or a difference of the size to or from a size of areference block (for example, LCU or SCU). For example, in a case oftransmitting information designating the block size as a syntax elementor the like, the information indirectly designating the size asdescribed above may be used as the information. This can sometimesdecrease an information amount of the information and improve encodingefficiency. In addition, this designation of the block size includesdesignation of a range of the block size (for example, designation of anallowable range of the block size).

<Control Over Computing Precision>

There has conventionally been known a method of controlling computingprecision of orthogonal transform and quantization (inverse quantizationand inverse orthogonal transform) on the basis of a bit depth ‘BitDepth’of an input image specified by a sequence parameter set (SPS) in imageencoding and decoding as described in, for example, NPL 1.

As depicted in, for example, A of FIG. 1, there is a method including,in encoding, obtaining (not depicted) a difference between an inputimage and a predicted image of the input image (prediction residual) andperforming primary horizontal transform 11, primary vertical transform12, secondary transform (not depicted), quantization 13, and encoding 14on the prediction residual res.

There is a method, in such encoding, including bit-shifting transformcoefficient data tmp by using a primary horizontal transform shiftamount fwdShift1 that is a predetermined scaling parameter in theprimary horizontal transform 11, bit-shifting transformed coefficientdata (also referred to as a “transform coefficient”) coef by using aprimary vertical transform shift amount fwdShift2 that is apredetermined scaling parameter in the primary vertical transform 12,and bit-shifting quantized coefficient data (also referred to as a“quantized coefficient”) qcoef by using a quantization shift amountfwdQShift that is a predetermined scaling parameter in the quantization13, thereby controlling the number of significant figures of each data.It is noted that, among these scaling parameters, the primary horizontaltransform shift amount fwdShift1 and the quantization shift amountfwdQShift depend on the bit depth ‘bitDepth’.

In the case of decoding corresponding to such encoding, as depicted in Bof FIG. 1, decoding 21, inverse quantization 22, inverse secondarytransform (not depicted), inverse primary vertical transform 23, andinverse primary horizontal transform 24 are sequentially performed on abit stream bitstream, a predicted image is added (not depicted) to anobtained prediction residual res_(rec), and a decoded image is obtained.

There is a method including, in such decoding, bit-shifting inverselyquantized coefficient data (that is, transform coefficient) coef byusing an inverse quantization shift amount invQShift that is apredetermined scaling parameter in the inverse quantization 22,bit-shifting transform coefficient data tmp by using an inverse primaryvertical transform shift amount invShift1 that is a predeterminedscaling parameter in the inverse primary vertical transform 23, andbit-shifting transformed residual data (that is, prediction residual)res_(rec) by using an inverse primary horizontal transform shift amountinvShift2 that is a predetermined scaling parameter as the inverseprimary horizontal transform 24, thereby controlling the number ofsignificant figures of each data. It is noted that, among these scalingparameters, the inverse primary horizontal transform shift amountinvShift2 and the inverse quantization shift amount invQShift depend onthe bit depth ‘bitDepth’.

In other words, in a case of supposing a narrow-range signal, data isnot computed using a full range from a minimum value coefMin to amaximum value coefMax that can be taken on by the data. In other words,as depicted in FIG. 2, bit fields (shaded parts in FIG. 2) used innumerical representation of a coefficient and a bit field (blank part inFIG. 2) not used in numerical representation are present in a bit width(log 2TransformRange+1) bit from an LSB (Least Significant Bit) to anMSB (Most Significant Bit) of data. That is, there is room for usingpart of the unused bit field as bits for increasing the number ofsignificant figures of the coefficient within the range from the minimumvalue coefMin to the maximum value coefMax.

In the example of FIG. 1, bit-shifting the coefficient data by usingthis room (unused range) makes it possible to control the number ofsignificant figures of each coefficient data (i.e., computing precision)in orthogonal transform and quantization (inverse quantization andinverse orthogonal transform). In a case of decoding, for example, thenumber of significant figures of the coefficient data or the residualdata is expanded in each of the inverse quantization processing, theinverse vertical transform processing, and inverse horizontal transformprocessing, as depicted in FIG. 3. Expanding the number of significantfigures of data enables improvement in computing precision.

However, a bit depth of an input image changes in a time direction or aspatial direction within a sequence, for example, per picture or perlocal area within each picture. For example, in a frame 41 depicted in Aof FIG. 4, an overall image is dark because of less illumination. Thedistribution of pixel values within this frame 41 is like a histogram42, and a signal range (width from a minimum value to a maximum value ofpixel values) (hereinafter, also referred to as a “bit depth”) is ninebits.

A frame 43 depicted in B of FIG. 4 is a next frame of the frame 41. Theframe 43 is an overall bright image because of much illumination. Thedistribution of pixel values of the frame 43 is like a histogram 44, anda bit depth is 10 bits.

In such way, for example, when a variation in illumination or the likeoccurs, the signal range (bit depth) changes between pictures. In otherwords, a change in the bit depth in the time direction also possiblyoccurs within a sequence.

In addition, the bit depth also possibly changes per local area withineach picture. In other words, a change in the bit depth in the spatialdirection also possibly occurs. For example, in a case of segmenting apicture 45 of a luminance signal depicted in FIG. 5 into 4×4=16 andsetting the segmented parts as local areas (local areas 00 to 33), ahistogram of pixel values of each local area of the picture 45 of thisluminance signal is as that of FIG. 6. In other words, a bit depth ofeach of the local areas 30 and 33 of the picture 45 is nine bits, and abit depth of each of an overall picture (all) and the other local areasis 10 bits. In such way, the bit depth of the luminance signal possiblychanges in the spatial direction according to a luminance distributionwithin the picture.

Furthermore, in a case of segmenting, for example, a picture 46 of achrominance signal (Cb) depicted in FIG. 7 into 4×4=16 and setting thesegmented parts as local areas (local areas 00 to 33), a histogram ofpixel values of each local area of the picture 46 of this chrominancesignal is as that depicted in FIG. 8. In other words, in this case, thelocal areas where a bit depth is seven bits, the local areas where a bitdepth is eight bits, and the local areas where a bit depth is nine bitsare present in the picture 46. In such way, the bit depth of thechrominance signal possibly changes in the spatial direction accordingto a chrominance distribution within the picture.

On the other hand, the scaling parameters used in the orthogonaltransform and the quantization (the inverse quantization and the inverseorthogonal transform) are set in the sequence parameter set as describedabove. In other words, control over the numbers of significant figuresin the orthogonal transform and the quantization (the inversequantization and the inverse orthogonal transform) is exercised persequence.

Owing to this, it is difficult to control the computing precision of theorthogonal transform and the quantization (the inverse quantization andthe inverse orthogonal transform) in the time direction or the spatialdirection within each sequence according to the change in the bit depthof the input image. Owing to this, there is a concern of a reduction inencoding efficiency.

2. Common Concept

<Control Over Computing Precision within Sequence>

To address the challenge, the number of significant figures of aprediction residual of an image is expanded on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of the image, the number of significantfigures of a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded is normalized on the basis ofthe bit depth of the local level, the quantized coefficient the numberof significant figures of which is normalized is encoded, and a bitstream is generated.

For example, an image processing apparatus includes an expansion sectionthat expands the number of significant figures of a prediction residualof an image on the basis of a bit depth indicating a range of pixelvalues of a local level that is a data unit smaller than a sequencelevel of the image, a normalization section that normalizes the numberof significant figures of a quantized coefficient obtained byperformance of orthogonal transform and quantization on the predictionresidual the number of significant figures of which is expanded by theexpansion section, on the basis of the bit depth of the local level, andan encoding section that encodes the quantized coefficient the number ofsignificant figures of which is normalized by the normalization sectionand that generates a bit stream.

By doing so, it is possible to perform orthogonal transform andquantization in a state in which the number of significant figures ofeach coefficient is controlled in the time direction or the spatialdirection even within each sequence. In other words, it is possible tocontrol computing precision of the orthogonal transform and thequantization in the time direction or the spatial direction even withinthe sequence. It is therefore possible to suppress a reduction inencoding efficiency.

Furthermore, a bit stream is decoded, the number of significant figuresof a quantized coefficient obtained by decoding of the bit stream isexpanded on the basis of a bit depth indicating a range of pixel valuesof a local level that is a data unit smaller than a sequence level, andthe number of significant figures of residual data obtained byperformance of inverse quantization and inverse orthogonal transform onthe quantized coefficient the number of significant figures of which isexpanded is normalized on the basis of the bit depth of the local level.

For example, an image processing apparatus includes a decoding sectionthat decodes a bit stream, an expansion section that expands the numberof significant figures of a quantized coefficient obtained by decodingof the bit stream by the decoding section, on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level, and a normalization section thatnormalizes the number of significant figures of residual data obtainedby performance of inverse quantization and inverse orthogonal transformon the quantized coefficient the number of significant figures of whichis expanded by the expansion section, on the basis of the bit depth ofthe local level.

By doing so, it is possible to perform inverse quantization and inverseorthogonal transform in a state in which the number of significantfigures of each coefficient is controlled in the time direction or thespatial direction even within each sequence. In other words, it ispossible to control the computing precision of the inverse quantizationand the inverse orthogonal transform in the time direction or thespatial direction even within the sequence. It is therefore possible tosuppress a reduction in encoding efficiency.

Furthermore, the number of significant figures of a prediction residualof an image is expanded on the basis of a bit depth indicating a rangeof pixel values of a local level that is a data unit smaller than asequence level of a predicted image corresponding to the image (or of adecoded image referred to at a time of generation of the predictedimage), the number of significant figures of a quantized coefficientobtained by performance of orthogonal transform and quantization on theprediction residual the number of significant figures of which isexpanded is normalized on the basis of the bit depth of the local levelof the predicted image (or of the decoded image referred to at the timeof generation of the predicted image), the quantized coefficient thenumber of significant figures of which is normalized is encoded, and abit stream is generated.

For example, an image encoding apparatus includes an expansion sectionthat expands the number of significant figures of a prediction residualof an image on the basis of a bit depth indicating a range of pixelvalues of a local level that is a data unit smaller than a sequencelevel of a predicted image corresponding to the image (or of a decodedimage referred to at a time of generation of the predicted image), anormalization section that normalizes the number of significant figuresof a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level of the predicted image (orof the decoded image referred to at the time of generation of thepredicted image), and an encoding section that encodes the quantizedcoefficient the number of significant figures of which is normalized bythe normalization section and that generates a bit stream.

By doing so, it is possible to perform orthogonal transform andquantization in a state in which the number of significant figures ofeach coefficient is controlled in the time direction or the spatialdirection even within each sequence. In other words, it is possible tocontrol computing precision of the orthogonal transform and thequantization in the time direction or the spatial direction even withinthe sequence. It is therefore possible to suppress a reduction inencoding efficiency.

Moreover, a bit stream is decoded, the number of significant figures ofa quantized coefficient obtained by decoding of the bit stream isexpanded on the basis of a bit depth indicating a range of pixel valuesof a local level that is a data unit smaller than a sequence level of apredicted image (or of a decoded image referred to at a time ofgeneration of the predicted image), and the number of significantfigures of residual data obtained by performance of inverse quantizationand inverse orthogonal transform on the quantized coefficient the numberof significant figures of which is expanded is normalized on the basisof the bit depth of the local level of the predicted image (or of thedecoded image referred to at the time of generation of the predictedimage).

For example, an image processing apparatus includes a decoding sectionthat decodes a bit stream, an expansion section that expands the numberof significant figures of a quantized coefficient obtained by decodingof the bit stream by the decoding section, on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of a predicted image (or of a decodedimage referred to at a time of generation of the predicted image), and anormalization section that normalizes the number of significant figuresof residual data obtained by performance of inverse quantization andinverse orthogonal transform on the quantized coefficient the number ofsignificant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level of the predicted image (orof the decoded image referred to at the time of generation of thepredicted image).

By doing so, it is possible to control the number of significant figuresof each data in the inverse quantization and the inverse orthogonaltransform in the time direction or the spatial direction even withineach sequence. In other words, it is possible to control the computingprecision of the inverse quantization and the inverse orthogonaltransform in the time direction or the spatial direction even within thesequence. It is therefore possible to suppress a reduction in encodingefficiency.

<Examples of Control>

FIG. 9 depicts a list of examples of control over the computingprecision described above. In a table depicted in FIG. 9, in a case ofan example of a first stage from the top (method #1) without countingthe row indicating the item names in an uppermost stage, an encodingside derives extension bit precision dBD that is a parameter indicatinga correction amount of control over the number of significant figuresfrom an input image. In addition, the control over (expansion andnormalization of) the number of significant figures with use of theextension bit precision dBD is performed separately (in isolation) fromthe orthogonal transform processing and the quantization processing andfrom the inverse quantization processing and the inverse orthogonaltransform processing. More specifically, the number of significantfigures of a prediction residual and that of a quantized coefficient areexpanded using this extension bit precision dBD before such series ofprocessing is performed. Furthermore, the coefficient data and theresidual data are normalized using this extension bit precision dBDafter such series of processing is performed.

The encoding side then transmits a difference parameter deltaX using theextension bit precision dBD to a decoding side. The decoding sidederives the extension bit precision dBD by using the transmitteddifference parameter deltaX. In addition, control over (expansion andnormalization of) the number of significant figures with use of theextension bit precision dBD is exercised (performed) separately (inisolation) from the inverse quantization processing and the inverseorthogonal transform processing. More specifically, the number ofsignificant figures of the quantized coefficient is expanded using thisextension bit precision dBD before such series of processing isperformed. Furthermore, the residual data is normalized using thisextension bit precision dBD after such series of processing isperformed.

In the case of the method #1, it is therefore possible to improve thecomputing precision in the orthogonal transform processing and thequantization processing and in the inverse quantization processing andthe inverse orthogonal transform processing (that is, it is possible tosuppress a reduction in computing precision). It is therefore possibleto improve encoding efficiency (that is, it is possible to suppress areduction in encoding efficiency).

Furthermore, control over (expansion and normalization of) the number ofsignificant figures with use of the extension bit precision dBD can beexercised as processing different (performed in isolation) from theorthogonal transform processing and the quantization processing and fromthe inverse quantization processing and the inverse orthogonal transformprocessing. It is therefore possible to realize such encoding usingconventional processing sections (such as an orthogonal transformsection, a quantization section, an inverse quantization section, and aninverse orthogonal transform section); thus, it is possible to suppressa growth of a load of processing related to encoding and decoding (forexample, an implementation cost of a circuit scale and the like, anddifficulty of design and development).

Moreover, the encoding side can control the extension bit precision dBDused on the decoding side since the difference parameter deltaX istransmitted from the encoding side to the decoding side. In other words,it is possible to improve a degree of freedom of processing on theencoding side, for example, it is possible for the encoding side tocontrol the computing precision (the number of significant figures ofdata) on the decoding side.

Moreover, in a case of an example of the second stage from the top ofthe table of FIG. 9 (method #2), both the encoding side and the decodingside derive the extension bit precision dBD from a predicted image (or adecoded image referred to at a time of generation of the predictedimage). In other words, in this case, the transmission of the differenceparameter deltaX is omitted. In addition, similarly to the method #1,control over (expansion and normalization of) the number of significantfigures with use of the extension bit precision dBD is exercised(performed) separately (in isolation) from the orthogonal transformprocessing and the quantization processing and from the inversequantization processing and the inverse orthogonal transform processing.More specifically, the number of significant figures of the predictionresidual and that of the quantized coefficient are expanded using thisextension bit precision dBD before such series of processing isperformed. Furthermore, the coefficient data and the residual data arenormalized using this extension bit precision dBD after such series ofprocessing is performed.

In the case of the method #2, it is therefore possible to improve thecomputing precision in the orthogonal transform processing and thequantization processing and in the inverse quantization processing andthe inverse orthogonal transform processing (that is, it is possible tosuppress a reduction in computing precision). It is therefore possibleto improve encoding efficiency (that is, it is possible to suppress areduction in encoding efficiency).

Furthermore, control over (expansion and normalization of) the number ofsignificant figures with use of the extension bit precision dBD can beexercised as processing different from (performed in isolation from) theorthogonal transform processing and the quantization processing and fromthe inverse quantization processing and the inverse orthogonal transformprocessing. It is therefore possible to realize such encoding by usingconventional processing sections (such as the orthogonal transformsection, the quantization section, the inverse quantization section, andthe inverse orthogonal transform section); thus, it is possible tosuppress an increase of difficulty of design and development.

Furthermore, since the transmission of the difference parameter deltaXis omitted as described above, it is possible to improve the encodingefficiency correspondingly (it is possible to reduce overhead related tothe extension bit precision dBD).

<Outline of Method #1>

In the case of, for example, the method #1, a series of processing fromthe primary horizontal transform 11 to the encoding 14 in encoding isperformed as depicted in FIG. 10.

In the primary horizontal transform 11, primary horizontal transformprocessing is performed, and the number of significant figures of thecoefficient data tmp is controlled using the primary horizontaltransform shift amount fwdShift1 of a sequence level. More specifically,the coefficient data tmp is bit-shifted by fwdShift1 bits (>>fwdShift1).

In the primary vertical transform 12, primary vertical transformprocessing is performed, and the number of significant figures of thetransform coefficient coef is controlled using the primary verticaltransform shift amount fwdShift2 of the sequence level. Morespecifically, the transform coefficient coef is bit-shifted by fwdShift2(>>fwdShift2).

In the quantization 13, quantization processing is performed, and thenumber of significant figures of the quantized coefficient qcoef iscontrolled using the quantization shift amount fwdQShift of the sequencelevel. More specifically, the quantized coefficient qcoef is bit-shiftedby fwdQShift bits (>>fwdQShift).

In the encoding 14, the quantized coefficient qcoef is encoded, and abit stream bitstream is obtained.

Further, per-area pixel minimum value/maximum value search 51 that isprocessing performed on an input image (Input sequence) for searching aminimum value (minPixelVal) and a maximum value (maxPixelVal) of pixelvalues per local level (block), which is a data unit smaller than asequence level, is performed.

In addition, dBD derivation 52 for deriving the extension bit precisiondBD per local level is performed using the minimum value (minPixelVal)and the maximum value (maxPixelVal) of the pixel values per local leveldetected by this search. In the dBD derivation 52, for example, eachlocal-level bit depth LocalBitDepth is derived by use of the minimumvalue (minPixelVal) and the maximum value (maxPixelVal) of pixel valuesper local level, as represented by the following Equation (1).

It is noted that the local bit depth LocalBitDepth derived in Equation(1) may further be corrected to LocalBitDepth=max(minLocalBitDepth,LocalBitDepth) by referring to a predetermined thresholdminLocalBitDepth (for example, eight), as represented in Equation (1A).By doing so, it is possible to suppress occurrence of a case in whichthe extension bit precision dBD derived in subsequent Equation (2)becomes excessively large in value (that is, the number of significantfigures is excessively expanded, and intermediate coefficient dataexceeds the range from the minimum value coefMin to the maximum valuecoefMax). It is noted that in Equation (1A), an operator max (x, y) isan operator that returns a larger value out of numeric values x and y.LocalBitDepth=ceil(log 2(maxPixelVal−minPixelVal))  (1)LocalBitDepth=max(minLocalBitDepth,LocalBitDepth)   (1A)

It is noted that log 2(x) is a function that returns a base 2logarithmic value of a real number x, and ceil(x) is a ceiling functionthat returns a minimum integer equal to or greater than x to the realnumber x. In addition, the extension bit precision dBD is derived by useof the local-level bit depth LocalBitDepth and a sequence-level bitdepth channelBitDepth, as represented by the following Equation (2).dBD=channelBitDepth−LocalBitDepth  (2)

It is noted that the controlled variable (shift amount, scalingparameter) of the number of significant figures determined by referringto the sequence-level bit depth channelBitDepth, as a parameter, will bereferred to as a “controlled variable (shift amount, scaling parameter)of the number of significant figures of the sequence level,” and thecontrolled variable (shift amount, scaling parameter) of the number ofsignificant figures of the sequence level corrected by the extension bitprecision dBD described above will be referred to as a “correctedcontrolled variable (shift amount, scaling parameter),” for the sake ofconvenience.

Furthermore, deltaX derivation 53 in which the difference parameterdeltaX is derived by use of the extension bit precision dBD derived inthe dBD derivation 52 is performed. In the deltaX derivation 53, forexample, a slice-level bit depth sliceBitDepth is derived by use of aminimum value (slice_min_val) and a maximum value (slice_max_val) ofpixel values per slice level, as represented by the following Equation(3).sliceBitDepth=ceil(log 2(slice_max_val−slice_min_val))  (3)

In addition, the difference parameter deltaX is derived by use of theextension bit precision dBD, the slice-level bit depth sliceBitDepth,and the sequence-level bit depth channelBitDepth, as represented by thefollowing Equation (4).deltaX=dBD+sliceBitDepth−channelBitDepth  (4)

Furthermore, expansion 54 that is processing for controlling the numberof significant figures of a prediction residual resi is performed usingthe extension bit precision dBD derived in the dBD derivation 52. Forexample, in this expansion 54, a prediction residual resiS is derived bybit-shifting the prediction residual resi to the left by extension bitprecision dBD bits (<<dBD). In other words, the number of significantfigures of the coefficient data is expanded by this expansion 54 beforeorthogonal transform and quantization are performed. In the primaryhorizontal transform processing 11, primary horizontal transform isperformed on this prediction residual resiS.

Furthermore, normalization 55 that is processing for controlling thenumber of significant figures of the quantized coefficient qcoef isperformed using the extension bit precision dBD derived in the dBDderivation 52. For example, in this normalization 55, the quantizedcoefficient qcoef is bit-shifted to the right by extension bit precisiondBD bits (>>dBD). In other words, the number of significant figures ofthe coefficient data having been subjected to orthogonal transform andquantization is normalized by this normalization 55. In the encoding 14,the quantized coefficient qcoef normalized in such way is encoded.

In addition, in the encoding 14, the difference parameter deltaX derivedin the deltaX derivation 53 is encoded and contained in a bit stream (abit stream containing the difference parameter deltaX is generated). Inother words, the difference parameter deltaX is transmitted from theencoding side to the decoding side. By doing so, the decoding side canalso derive the extension bit precision dBD by using this differenceparameter deltaX.

Furthermore, in the case of, for example, the method #1, a series ofprocessing from the decoding 21 to the inverse primary horizontaltransform 24 is performed in decoding, as depicted in FIG. 11.

In the decoding 21, the bit stream bitstream is decoded, and thequantized coefficient qcoef is obtained.

In the inverse quantization 22, inverse quantization processing isperformed, and the number of significant figures of the transformcoefficient coefS′ is controlled using the inverse quantization shiftamount invQShift of the sequence level. More specifically, the transformcoefficient coef is bit-shifted by invQShift bits (>>invQShift).

In the inverse primary vertical transform 23, inverse primary verticaltransform processing is performed, and the number of significant figuresof the coefficient data tmp is controlled by use of an inverse primaryvertical transform shift amount invShift1 of the sequence level. Morespecifically, the coefficient data tmp is bit-shifted by invShift1 bits(>>invShift1).

In the inverse primary horizontal transform 24, inverse primaryhorizontal transform processing is performed, and the number ofsignificant figures of a prediction residual resiS' is controlled usingthe inverse primary horizontal transform shift amount invShift2 of thesequence level. More specifically, the prediction residual resiS' isbit-shifted by invShift2 bits (>>invShift2).

Moreover, in the decoding 21, the sequence-level bit depthchannelBitDepth and the slice-level bit depth sliceBitDepth are derived,and further, the difference parameter deltaX is obtained.

By use of the sequence-level bit depth channelBitDepth, the slice-levelbit depth sliceBitDepth, and the difference parameter deltaX, dBDderivation 61 in which the extension bit precision dBD is derived isperformed. In the dBD derivation 61, the extension bit precision dBD isderived using those parameters as represented by the following Equation(5).dBD=channelBitDepth−sliceBitDepth+deltaX  (5)

Further, expansion 62 that is processing for controlling the number ofsignificant figures of the quantized coefficient qcoef is performedusing the extension bit precision dBD derived in the dBD derivation 61.For example, in this expansion 62, a quantized coefficient qcoefS′ isderived by bit-shifting the quantized coefficient qcoef to the left byextension bit precision dBD bits (<<dBD). In other words, the number ofsignificant figures of the coefficient data is expanded by thisexpansion 62 before inverse quantization and inverse orthogonaltransform are performed. In the inverse quantization 22, this quantizedcoefficient qcoefS′ the number of significant figures of which isexpanded is inversely quantized.

Further, normalization 63 that is processing for controlling the numberof significant figures of a prediction residual resiS' is performedusing the extension bit precision dBD derived in the dBD derivation 61.For example, in this normalization 63, the prediction residual resiS' isbit-shifted to the right by extension bit precision dBD bits (<<dBD). Inother words, the number of significant figures of the coefficient datasubjected to inverse quantization and inverse orthogonal transform isnormalized by this normalization 63.

As described above, the expansion 62 and the normalization 63 controlthe number of significant figures by using the local-level extension bitprecision dBD; thus, it is possible to perform such expansion of(control over) the number of significant figures according to the changeof the bit depth within a sequence in the time direction or the spatialdirection. In the case of, for example, decoding, it is thereforepossible to perform each of the inverse quantization processing, theinverse vertical transform processing, and the inverse horizontaltransform processing in a state in which the number of significantfigures of each coefficient is further expanded from expansion of thesequence level (FIG. 3), as depicted in FIG. 12. By expanding the numberof significant figures, the computing precision is improved. It istherefore possible to suppress a reduction in encoding efficiency.

Likewise, in the case of encoding described above, the number ofsignificant figures can be controlled according to the change of the bitdepth in the time direction or the spatial direction within thesequence; thus, the number of significant figures of each coefficientcan be further expanded from expansion of the sequence level, in each ofthe horizontal transform processing, the vertical transform processing,and the quantization processing. It is therefore possible to improvecomputing precision and suppress a reduction in encoding efficiency.

It is noted that in the case of encoding, similarly to the case ofdecoding, the number of significant figures is controlled in each of theinverse quantization processing, the inverse vertical transformprocessing, and the inverse horizontal transform processing. In otherwords, it is possible to expand the number of significant figuressimilarly to the case of decoding. In other words, it is possible toimprove encoding efficiency of the series of processing and suppress areduction in encoding efficiency.

<Outline of Method #2>

In the case of, for example, encoding by the method #2 of FIG. 9, aseries of processing from the primary horizontal transform 11 to theencoding 14 is performed, as depicted in FIG. 13.

Moreover, in prediction 71, a predicted image corresponding to an inputimage is generated. Further, in computing 72, a difference (predictionresidual resi) between the input image and the predicted image generatedby the prediction 71 is obtained.

In the case of this method #2, the per-area pixel minimum value/maximumvalue search 51 is performed on the predicted image obtained by theprediction 71. In other words, by the per-area pixel minimumvalue/maximum value search 51, a minimum value (minPredPixelVal) and amaximum value (maxPredPixelVal) of pixel values per local level (block)of the predicted image are searched.

In addition, in the dBD derivation 52, the extension bit precision dBDis derived using the minimum value (minPredPixelVal) and the maximumvalue (maxPredPixelVal) of the pixel values per local level of thepredicted image detected by this search. For example, each local-levelbit depth PredBitDepth of a predicted image is derived by use of theminimum value (minPredPixelVal) and the maximum value (maxPredPixelVal)of the pixel value at each local level of the predicted image, asrepresented by the following Equation (6).

It is noted that the local-level bit depth PredBitDepth derived inEquation (6) may further be corrected toPredBitDepth=max(minLocalBitDepth, PredBitDepth) by referring to apredetermined threshold minLocalBitDepth (for example, eight), asdepicted in Equation (6A). By doing so, it is possible to suppressoccurrence of a case in which the extension bit precision dBD derived insubsequent Equation (7) becomes excessively large in value (that is, thenumber of significant figures is excessively expanded, and intermediatecoefficient data exceeds the range from the minimum value coefMin to themaximum value coefMax). It is noted that in Equation (6A), the operatormax (x, y) is an operator that returns a larger value out of numericvalues x and y.PredBitDepth=ceil(log 2(maxPredPixelVal−minPredPixelVal))  (6)PredBitDepth=max(minLocalBitDepth,PredBitDepth)   (6A)

In addition, the extension bit precision dBD is derived as representedby the following Equation (7), by using the local-level bit depthPredBitDepth of the predicted image and the sequence-level bit depthchannelBitDepth.dBD=channelBitDepth−PredBitDepth  (7)

In such way, the expansion 54 that is processing for controlling thenumber of significant figures of the prediction residual resi isperformed using the extension bit precision dBD derived in the dBDderivation 52. For example, in this expansion 54, a prediction residualresiS is derived by bit-shifting the prediction residual resi to theleft by extension bit precision dBD bits (<<dBD). In other words, thenumber of significant figures of the coefficient data is expanded bythis expansion 54 before orthogonal transform and quantization areperformed. In the primary horizontal transform processing 11, primaryhorizontal transform is performed on this prediction residual resiS.

Further, the normalization 55 that is processing for controlling thenumber of significant figures of a quantized coefficient qcoefS isperformed using the extension bit precision dBD derived in the dBDderivation 52. For example, in this normalization 55, the quantizedcoefficient qcoefS is bit-shifted to the right by extension bitprecision dBD bits (>>dBD). In other words, the number of significantfigures of the coefficient data having been subjected to orthogonaltransform and quantization is normalized by this normalization 55. Inthe encoding 14, the quantized coefficient qcoef obtained bynormalization of the quantized coefficient qcoefS in such way isencoded.

It is noted, however, that in the case of this method #2, the processingof the deltaX derivation 53 is omitted, and the processing related toencoding of the difference parameter deltaX is omitted in the encoding14. In other words, in the case of the method #2, transmission of thedifference parameter deltaX from the encoding side to the decoding sideis omitted.

Moreover, in the case of, for example, decoding in the method #2, aseries of processing from the decoding 21 to the inverse primaryhorizontal transform 24 is performed as depicted in FIG. 14, similarlyto the case of the method #1. Furthermore, in prediction 81, a predictedimage corresponding to an input image is generated.

In the case of this method #2, per-area pixel minimum value/maximumvalue search 83 that is processing for searching a minimum value(minPredPixelVal) and a maximum value (maxPredPixelVal) of pixel valuesper local level (block) that is a data unit smaller than the sequencelevel is performed on the predicted image obtained by the prediction 81.

Furthermore, in the dBD derivation 61, the extension bit precision dBDis derived using the minimum value (minPredPixelVal) and the maximumvalue (maxPredPixelVal) of the pixel values per local level of thepredicted image detected by this search. For example, each local-levelbit depth PredBitDepth of the predicted image is derived using theminimum value (minPredPixelVal) and the maximum value (maxPredPixelVal)of pixel values per local level of the predicted image, as representedby the above Equation (6).

In addition, the extension bit precision dBD is derived as representedby the above Equation (7) by using the local-level bit depthPredBitDepth of the predicted image and the sequence-level bit depthchannelBitDepth.

In the expansion 62, the number of significant figures of the quantizedcoefficient qcoef is controlled using the extension bit precision dBDderived in the dBD derivation 61 in such way. For example, in thisexpansion 62, a quantized coefficient qcoefS′ is derived by bit-shiftingthe quantized coefficient qcoef to the left by extension bit precisiondBD bits (<<dBD). In other words, the number of significant figures ofthe coefficient data is expanded by this expansion 62 before inversequantization and orthogonal transform are performed. In the inversequantization 22, this quantized coefficient qcoefS′ is inverselyquantized.

In the normalization 63, the number of significant figures of thequantized coefficient qcoefS is controlled using the extension bitprecision dBD derived in the dBD derivation 61. For example, in thisnormalization 63, the prediction residual resiS' is bit-shifted to theright by extension bit precision dBD bits (>>dBD). In other words, thenumber of significant figures of the coefficient data having beensubjected to inverse quantization and inverse orthogonal transform isnormalized by this normalization 63. In computing 82, the predictionresidual resi′ obtained by normalization of the prediction residualresiS′ in such way and the predicted image generated in the prediction81 are added up, and a reconstructed image rec′ is obtained.

As described above, the expansion 62 and the normalization 63 controlthe number of significant figures by using the local-level extension bitprecision dBD; thus, it is possible to perform such expansion of(control over) the number of significant figures according to the changeof the bit depth in the time direction or the spatial direction within asequence. In the case of, for example, decoding, it is thereforepossible to perform each of the inverse quantization processing, theinverse vertical transform processing, and the inverse horizontaltransform processing in a state in which the number of significantfigures of each coefficient is further expanded from expansion of thesequence level (FIG. 3), as depicted in FIG. 12. By expanding the numberof significant figures, the computing precision is improved. It istherefore possible to suppress a reduction in encoding efficiency.

Likewise, in the case of encoding described above, the number ofsignificant figures can be controlled according to the change of the bitdepth in the time direction or the spatial direction within thesequence; thus, the number of significant figures of each coefficientcan be further expanded from expansion of the sequence level, in each ofthe horizontal transform processing, the vertical transform processing,and the quantization processing. It is therefore possible to improvecomputing precision and suppress a reduction in encoding efficiency.

It is noted that in the case of encoding, similarly to the case ofdecoding, the number of significant figures is controlled in each of theinverse quantization processing, the inverse vertical transformprocessing, and the inverse horizontal transform processing. In otherwords, it is possible to expand the number of significant figuressimilarly to the case of decoding. In other words, it is possible toimprove encoding efficiency of the series of processing and suppress areduction in encoding efficiency.

It is noted that in the method #2, the local-level bit depth of a blockto be processed is derived by Equations (6) and (6A) by reference to thepredicted image corresponding to the block to be processed obtained inthe prediction 71 (81) as depicted in FIG. 13 (FIG. 14). Alternatively,the local bit depth of the block to be processed may be derived usingthe decoded image referred to at the time of generation of the predictedimage, as an alternative to the predicted image (method #2′).

In other words, in FIG. 13 (FIG. 14), the per-area pixel minimumvalue/maximum value search 51 (83) is performed on a decoded imageRec_(ref) referred to at the time of generation of the predicted imagein the prediction 71 (81), to search a minimum value (minRecPixelVal)and a maximum value (maxRecPixelVal) of pixel values per local level(block). Subsequently, the dBD derivation 52 (61) is performed using theminimum value (minRecPixelVal) and the maximum value (maxRecPixelVal) ofpixel values per local level of the decoded image detected by thissearch. In the dBD derivation 52 (61), each local-level bit depthRecBitDepth of the decoded image is derived as represented by thefollowing Equations (6′) and (6A′). In addition, the extension bitprecision dBD is derived as represented by Equation (7′).RecBitDepth=ceil(log 2(maxRecPixelVal−minRecPixelVal))  (6′)RecBitDepth=max(minLocalBitDepth,recBitDepth)  (6A′)dBD=channelBitDepth−RecBitDepth  (7′)

By doing so, the method #2′ can exhibit advantageous effects similar tothose of the method of deriving the extension bit precision dBD byreferring to the predicted image (method #2). It is noted that theadvantage of the method #2′ over the method #2 is that the extension bitprecision dBD can be derived only from the decoded image without theneed to generate the predicted image.

It is noted that the decoded image Rec_(ref) in the above descriptionindicates a local decoded pixel area referred to for generation of anintra predicted image of a block to be processed in a case of intraprediction and indicates a local decoded pixel area referred to forgeneration of an inter predicted image of the block to be processed in acase of inter prediction.

3. First Embodiment

<Image Encoding Apparatus>

In the first embodiment, details of the method #1 described above willbe described. First, configurations of exercising such control over thecomputing precision at the time of encoding will be described. FIG. 15is a block diagram depicting an example of configurations of an imageencoding apparatus according to one aspect of an image processingapparatus to which the present technology is applied. An image encodingapparatus 100 depicted in FIG. 15 is an apparatus that encodes imagedata regarding an image sequence. For example, the image encodingapparatus 100 implements the technologies described in NPL 1 to NPL 3and encodes image data regarding an image sequence by a method compliantwith standards described in any of those documents.

It is noted that principal configurations such as processing sectionsand flows of data depicted in FIG. 15 are not necessarily allconfigurations. In other words, processing sections that are notdepicted as blocks in FIG. 15 may be present or processing and flows ofdata that are not indicated by arrows or the like in FIG. 15 may bepresent in the image encoding apparatus 100. This applies to otherdrawings for illustrating processing sections and the like within theimage encoding apparatus 100.

As depicted in FIG. 15, the image encoding apparatus 100 has a controlsection 101, a reordering buffer 111, a computing section 112, anorthogonal transform section 113, a quantization section 114, anencoding section 115, an accumulation buffer 116, an inversequantization section 117, an inverse orthogonal transform section 118, acomputing section 119, an in-loop filter section 120, a frame memory121, a prediction section 122, and a rate control section 123.

The image encoding apparatus 100 also has an expansion section 131, anormalization section 132, an expansion section 133, and a normalizationsection 134.

<Control Section>

The control section 101 segments image sequence data held in thereordering buffer 111 into blocks (such as CUs, PUs, or transformblocks) that are processing units, on the basis of an external blocksize or a block size of each processing unit designated in advance. Inaddition, the control section 101 determines encoding parameters (suchas header information Hinfo, prediction mode information Pinfo,transform information Tinfo, and filter information Finfo) to besupplied to the blocks, on the basis of, for example, RDO(Rate-Distortion Optimization).

Details of these encoding parameters will be described later. Upondetermining the encoding parameters described above, the control section101 supplies those encoding parameters to the blocks. Specifically, thecontrol section 101 supplies the encoding parameters as follows.

The header information Hinfo is supplied to each block.

The prediction mode information Pinfo is supplied to the encodingsection 115 and the prediction section 122.

The transform information Tinfo is supplied to the encoding section 115,the orthogonal transform section 113, the quantization section 114, theinverse quantization section 117, the inverse orthogonal transformsection 118, the expansion section 131, the normalization section 132,the expansion section 133, and the normalization section 134.

The filter information Finfo is supplied to the in-loop filter section120.

<Reordering Buffer>

Fields (input images) of the image sequence data are input to the imageencoding apparatus 100 in order of reproduction (order of display). Thereordering buffer 111 acquires and holds (stores) the input images inthe order of reproduction (order of display). The reordering buffer 111reorders the input images in order of encoding (order of decoding) orsegments each of the input images into blocks as processing units, undercontrol of the control section 101. The reordering buffer 111 supplieseach processed input image to the computing section 112. In addition,the reordering buffer 111 supplies the input image (original image) tothe prediction section 122 and the in-loop filter section 120.

<Computing Section>

An image I corresponding to each block as the processing unit and apredicted image P supplied from the prediction section 122 are input tothe computing section 112, and the computing section 112 subtracts thepredicted image P from an image rec as represented by the followingEquation (8), derives a prediction residual resi, and supplies theprediction residual resi to the expansion section 131.resi=rec−P  (8)<Orthogonal Transform Section>

The prediction residual resi supplied from the expansion section 131 andthe transform information Tinfo supplied from the control section 101are input to the orthogonal transform section 113, and the orthogonaltransform section 113 orthogonally transforms the prediction residualresi on the basis of the transform information Tinfo and derives thetransform coefficient coef. The orthogonal transform section 113supplies the obtained transform coefficient coef to the quantizationsection 114.

<Quantization Section>

The transform coefficient coef supplied from the orthogonal transformsection 113 and the transform information Tinfo supplied from thecontrol section 101 are input to the quantization section 114, and thequantization section 114 scales (quantizes) the transform coefficientcoef on the basis of the transform information Tinfo. It is noted that arate of this quantization is controlled by the rate control section 123.The quantization section 114 supplies the quantized transformcoefficient obtained by such quantization, that is, quantized transformcoefficient qcoefS, to the normalization section 132.

<Encoding Section>

The quantized coefficient qcoef supplied from the normalization section132, various encoding parameters (such as the header information Hinfo,the prediction mode information Pinfo, the transform information Tinfo,and the filter information Finfo) supplied from the control section 101,information associated with filters such as filter coefficients suppliedfrom the in-loop filter section 120, and information associated with anoptimum prediction mode supplied from the prediction section 122 areinput to the encoding section 115. The encoding section 115 performsvariable-length encoding (for example, arithmetic encoding) on thequantized coefficient qcoef and generates a bit sequence (encoded data).

In addition, the encoding section 115 derives residual information Rinfofrom the quantized coefficient qcoef, encodes the residual informationRinfo, and generates a bit sequence.

Further, the encoding section 115 contains the information associatedwith the filters supplied from the in-loop filter section 120 in thefilter information Finfo and contains the information associated withthe optimum prediction mode supplied from the prediction section 122 inthe prediction mode information Pinfo. The encoding section 115 thenencodes the various encoding parameters (such as the header informationHinfo, the prediction mode information Pinfo, the transform informationTinfo, and the filter information Finfo) described above and generatesbit sequences.

Moreover, the encoding section 115 multiplexes the bit sequences of thevarious kinds of information generated as described above and generatesencoded data. The encoding section 115 supplies the encoded data to theaccumulation buffer 116.

<Accumulation Buffer>

The accumulation buffer 116 temporarily holds the encoded data obtainedby the encoding section 115. The accumulation buffer 116 outputs theencoded data it holds to outside of the image encoding apparatus 100 as,for example, a bit stream at a predetermined timing. This encoded datais transmitted, for example, to a decoding side via a freely-selectedrecording medium, a freely-selected transmission medium, afreely-selected information processing apparatus, and the like. In otherwords, the accumulation buffer 116 also serves as a transmission sectionthat transmits the encoded data (bit stream).

<Inverse Quantization Section>

The inverse quantization section 117 performs processing associated withinverse quantization. For example, the quantized coefficient qcoefS′supplied from the expansion section 133 and the transform informationTinfo supplied from the control section 101 are input to the inversequantization section 117, and the inverse quantization section 117scales (inversely quantizes) a value of the quantized coefficient qcoefon the basis of the transform information Tinfo. It is noted that thisinverse quantization is inverse processing of the quantization performedby the quantization section 114. The inverse quantization section 117supplies transform coefficient coefS' obtained by such inversequantization to the inverse orthogonal transform section 118.

<Inverse Orthogonal Transform Section>

The inverse orthogonal transform section 118 performs processingassociated with inverse orthogonal transform. For example, the transformcoefficient coefS' supplied from the inverse quantization section 117and the transform information Tinfo supplied from the control section101 are input to the inverse orthogonal transform section 118, and theinverse orthogonal transform section 118 performs inverse orthogonaltransform on the transform coefficient coefS' on the basis of thetransform information Tinfo and derives a prediction residual resiS′. Itis noted that this inverse orthogonal transform is inverse processing ofthe orthogonal transform performed by the orthogonal transform section113. The inverse orthogonal transform section 118 supplies theprediction residual resiS′ obtained by such inverse orthogonal transformto the normalization section 134. It is noted that description to begiven (below) with respect to the decoding side can be applied to theinverse orthogonal transform section 118 since the inverse orthogonaltransform section 118 is similar to an inverse orthogonal transformsection on the decoding side (to be described below).

<Computing Section>

The prediction residual resi′ supplied from the normalization section134 and the predicted image P supplied from the prediction section 122are input to the computing section 119. The computing section 119 addsup the prediction residual resi′ and the predicted image P correspondingto the prediction residual resi′ and derives a local decoded imageR_(local). The computing section 119 supplies the derived local decodedimage R_(local) to the in-loop filter section 120 and the frame memory121.

<In-Loop Filter Section>

The in-loop filter section 120 performs processing associated within-loop filter processing. For example, the local decoded imageR_(local) supplied from the computing section 119, the filterinformation Finfo supplied from the control section 101, and the inputimage (original image) supplied from the reordering buffer 111 are inputto the in-loop filter section 120. It is noted that information to beinput to the in-loop filter section 120 can be any information, andinformation other than these pieces of information may be input to thein-loop filter section 120. For example, information regarding aprediction mode, motion information, a code amount target value, aquantization parameter QP, a picture type, blocks (CUs, CTUs, or thelike), and the like may be input to the in-loop filter section 120, asneeded.

The in-loop filter section 120 performs filter processing on the localdecoded image R_(local) on the basis of the filter information Finfo, asappropriate. The in-loop filter section 120 uses the input image(original image) and other pieces of input information in the filterprocessing, as needed.

For example, the in-loop filter section 120 applies four in-loop filterswhich are a bilateral filter, a deblocking filter (DBF (DeBlockingFilter)), an adaptive offset filter (SAO (Sample Adaptive Offset), andan adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Itis noted that the specific filters to be applied and order ofapplication thereof can be selected freely, and selection can be made asappropriate.

Needless to say, the filter processing to be performed by the in-loopfilter section 120 can be selected freely and is not limited to anexample described above. For example, the in-loop filter section 120 mayapply a Wiener filter and the like.

The in-loop filter section 120 supplies the filter-processed localdecoded image R_(local) to the frame memory 121. It is noted that in acase of transmitting, for example, the information associated withfilters such as filter coefficients to the decoding side, the in-loopfilter section 120 supplies the information associated with filters tothe encoding section 115.

<Frame Memory>

The frame memory 121 performs processing associated with storage of datarelated to images. For example, the local decoded image R_(local)supplied from the computing section 119 and the filter-processed localdecoded image R_(local) supplied from the in-loop filter section 120 areinput to the frame memory 121, and the frame memory 121 stores theimages. In addition, the frame memory 121 reconstructs a decoded image Rper picture unit by using the local decoded image R_(local) and holdsthe decoded image R (stores the decoded image R in a buffer within theframe memory 121). The frame memory 121 supplies the decoded image R (orpart of the decoded image R) to the prediction section 122 in responseto a request of the prediction section 122.

<Prediction Section>

The prediction section 122 performs processing associated withgeneration of a predicted image. For example, the prediction modeinformation Pinfo supplied from the control section 101, the input image(original image) supplied from the reordering buffer 111, and thedecoded image R (or part of the decoded image R) read out from the framememory 121 are input to the prediction section 122. The predictionsection 122 performs prediction processing such as inter prediction orintra prediction by using the prediction mode information Pinfo and theinput image (original image), performs prediction while referring to thedecoded image R as a reference image, performs motion compensationprocessing on the basis of a result of the prediction, and generates thepredicted image P. The prediction section 122 supplies the generatedpredicted image P to the computing sections 112 and 119. In addition,the prediction section 122 supplies a prediction mode selected by theprocessing described above, that is, the information associated with theoptimum prediction mode, to the encoding section 115, as needed.

<Rate Control Section>

The rate control section 123 performs processing associated with ratecontrol. For example, the rate control section 123 controls a rate of aquantization operation performed by the quantization section 114, on thebasis of a code amount of the encoded data accumulated in theaccumulation buffer 116, in such a manner as not to generate overflow orunderflow.

<Expansion Section>

The prediction residual resi supplied from the computing section 112 andthe transform information Tinfo supplied from the control section 101are input to the expansion section 131, and the expansion section 131performs processing associated with control over (expansion of) thenumber of significant figures of the prediction residual resi. Forexample, the expansion section 131 bit-shifts the prediction residualresi to the left by extension bit precision dBD bits (<<dBD) and expandsthe number of significant figures of the prediction residual resi. Inother words, the expansion section 131 expands the number of significantfigures of the prediction residual resi that is to be subjected toorthogonal transform and quantization, according to the change of thebit depth in the time direction or the spatial direction within eachsequence. Furthermore, the expansion section 131 supplies the predictionresidual resiS the number of significant figures of which is expandedand which is obtained by the bit-shifting, to the orthogonal transformsection 113.

<Normalization Section>

The quantized coefficient qcoefS supplied from the quantization section114 and the transform information Tinfo supplied from the controlsection 101 are input to the normalization section 132, and thenormalization section 132 performs processing associated with controlover (normalization of) the number of significant figures of thequantized coefficient qcoefS. For example, the normalization section 132bit-shifts the quantized coefficient qcoefS to the right by extensionbit precision dBD bits (>>dBD). In other words, the normalizationsection 132 normalizes the number of significant figures of thequantized coefficient qcoefS subjected to orthogonal transform andquantization, according to the change of the bit depth in the timedirection or the spatial direction within each sequence. Further, thenormalization section 132 supplies the quantized coefficient qcoef thenumber of significant figures of which is normalized and which isobtained by the bit-shifting, to the encoding section 115 and theexpansion section 133.

<Expansion Section>

The quantized coefficient qcoef supplied from the normalization section132 and the transform information Tinfo supplied from the controlsection 101 are input to the expansion section 133, and the expansionsection 133 performs processing associated with control over (expansionof) the number of significant figures of the quantized coefficientqcoef. For example, the expansion section 133 bit-shifts the quantizedcoefficient qcoef to the left by extension bit precision dBD bits(<<dBD) and expands the number of significant figures of the quantizedcoefficient qcoef. In other words, the expansion section 133 expands thenumber of significant figures of the quantized coefficient qcoef that isyet to be subjected to inverse quantization and inverse orthogonaltransform, according to the change of the bit depth in the timedirection or the spatial direction within each sequence. Further, theexpansion section 133 supplies the quantized coefficient qcoefS′ thenumber of significant figures of which is expanded and which is obtainedby the bit-shifting, to the inverse quantization section 117.

<Normalization Section>

The prediction residual resiS′ supplied from the inverse orthogonaltransform section 118 and the transform information Tinfo supplied fromthe control section 101 are input to the normalization section 134, andthe normalization section 134 performs processing associated withcontrol over (normalization of) the number of significant figures of theprediction residual resiS′. For example, the normalization section 134bit-shifts the prediction residual resiS′ to the right by extension bitprecision dBD bits (>>dBD) and normalizes the number of significantfigures of the prediction residual resiS′. In other words, thenormalization section 134 normalizes the number of significant figuresof the prediction residual resiS′ having been subjected to inversequantization and inverse orthogonal transform, according to the changeof the bit depth in the time direction or the spatial direction withineach sequence. Further, the normalization section 134 supplies theprediction residual resi′ the number of significant figures of which isnormalized and which is obtained by the bit-shifting, to the computingsection 119.

With the configurations described above, the image encoding apparatus100 can perform each of the orthogonal transform processing, thequantization processing, the inverse quantization processing, and theinverse orthogonal transform processing in a state in which the numberof significant figures of each coefficient is further expanded fromexpansion of the sequence level, and improve the computing precision ofthose series of processing. In other words, it is possible to suppress areduction in encoding efficiency.

<Details of Control Section>

FIG. 16 is a block diagram depicting an example of principalconfigurations of the control section 101, the principal configurationsbeing related to generation of information associated with control overthe number of significant figures. As depicted in FIG. 16, the controlsection 101 has a sequence bit depth setting section 151, a slice bitdepth setting section 152, a pixel minimum value/maximum value searchsection 153, a dBD derivation section 154, and a deltaX derivationsection 155. While the control section 101 performs processing otherthan the generation of the information associated with the control overthe number of significant figures such as generation of other pieces ofinformation as described above, description of configurations withrespect to those series of processing is omitted.

The input image from the reordering buffer 111 is input to the sequencebit depth setting section 151, and the sequence bit depth settingsection 151 sets the sequence bit depth channelBitDepth on the basis ofexternal parameters. The sequence bit depth setting section 151 suppliesthe set sequence bit depth channelBitDepth to the dBD derivation section154 and the deltaX derivation section 155. In addition, the sequence bitdepth setting section 151 supplies the sequence bit depthchannelBitDepth to the orthogonal transform section 113, thequantization section 114, the inverse quantization section 117, and theinverse orthogonal transform section 118, as the transform informationTinfo. Furthermore, the sequence bit depth setting section 151 generatesthe information associated with the sequence bit depth and supplies theinformation to the encoding section 115.

The input image from the reordering buffer 111 is input to the slice bitdepth setting section 152, and the slice bit depth setting section 152sets the slice bit depth sliceBitDepth. The slice bit depth settingsection 132 supplies the set slice bit depth sliceBitDepth to the deltaXderivation section 155. In addition, the slice bit depth setting section152 supplies the information associated with the slice bit depth to theencoding section 115.

The input image from the reordering buffer 111 is input to the pixelminimum value/maximum value search section 153, and the pixel minimumvalue/maximum value search section 153 searches the minimum value(minPixelVal) and the maximum value (maxPixelVal) of pixel values of theinput image per local level that is a data unit smaller than thesequence level. The pixel minimum value/maximum value search section 153supplies the minimum value (minPixelVal) and the maximum value(maxPixelVal) of each local level detected by the search to the dBDderivation section 154.

The sequence bit depth channelBitDepth supplied from the sequence bitdepth setting section 151 and the minimum value (minPixelVal) and themaximum value (maxPixelVal) supplied from the pixel minimumvalue/maximum value search section 153 are input to the dBD derivationsection 154, and the dBD derivation section 154 derives the extensionbit precision dBD per local level on the basis of those parameters.

The dBD derivation section 154 supplies the derived extension bitprecision dBD to the deltaX derivation section 155. In addition, the dBDderivation section 154 supplies the derived extension bit precision dBDto the orthogonal transform section 113, the quantization section 114,the inverse quantization section 117, the inverse orthogonal transformsection 118, the expansion section 131, the normalization section 132,the expansion section 133, and the normalization section 134, as thetransform information Tinfo.

The extension bit precision dBD supplied from the dBD derivation section154 is input to the deltaX derivation section 155, and the deltaXderivation section 155 derives the difference parameter deltaX. ThedeltaX derivation section 155 supplies the derived difference parameterdeltaX to the encoding section 115 as the information associated withthe extension bit precision.

<Flow of Image Encoding Processing>

A flow of each processing executed by the image encoding apparatus 100described above will next be described. An example of a flow of imageencoding processing will first be described with reference to aflowchart of FIG. 17.

When the image encoding processing is started, the reordering buffer 111is controlled by the control section 101 to reorder input frames of theimage sequence data from the order of display to the order of encodingin Step S101.

In Step S102, the control section 101 sets processing units to eachinput image held by the reordering buffer 111 (segments each input imageinto blocks).

In Step S103, the control section 101 determines (sets) encodingparameters for each input image held by the reordering buffer 111.

In Step S104, the prediction section 122 performs prediction processingand generates a predicted image or the like in an optimum predictionmode. For example, in this prediction processing, the prediction section122 performs intra prediction to generate a predicted image or the likein an optimum intra prediction mode, performs inter prediction togenerate a predicted image or the like in an optimum inter predictionmode, and selects the optimum prediction mode on the basis of a costfunction value and the like from them.

In Step S105, the computing section 112 computes a difference betweenthe input image and the predicted image in the optimum mode selected bythe prediction processing of Step S104. In other words, the computingsection 112 generates the prediction residual resi between the inputimage and the predicted image. Data volume of the prediction residualresi′ obtained in such way is reduced, compared with original imagedata. Therefore, it is possible to compress the data volume, comparedwith a case of encoding an image as it is.

In Step S106, the expansion section 131 expands the prediction residualresi of a component X generated by the processing of Step S105, by usingthe extension bit precision dBD per extension bit precision controlgroup SG by the following Equation (9), and obtains the expandedprediction residual resiS.At dBD>0, resiS=resi<<dBDAt dBD=0, resiS=resi  (9)

In Step S107, the orthogonal transform section 113 performs orthogonaltransform processing on the prediction residual resiS generated by theprocessing of Step S106 and derives a transform coefficient coefS.

In Step S108, the quantization section 114 quantizes the transformcoefficient coefS obtained by the processing of Step S106 by, forexample, using the quantization parameter calculated by the controlsection 101 and derives the quantized coefficient qcoefS.

In Step S109, the normalization section 132 normalizes the quantizedcoefficient qcoefS of the component X obtained by the processing of StepS108, by using the extension bit precision dBD per extension bitprecision control group SG as represented by the following Equation(10), and obtains the normalized quantized coefficient qcoef.At dBD>0,qcoef=(coefS+offsetBD)>>dBDwhere offsetBD=1<<(dBD−1)At dBD=0,qcoef=qcoefS  (10)

Normalization may be performed by the following Equation (11) as analternative to this Equation (10).At dBD>0,qcoef=sign(qcoefS)*(abs(qcoefS)+offsetBD)>>dBDAt dBD=0,qcoef=qcoefS  (11)

In the Equation, sign(x) is an operator that returns a positive ornegative sign of a real number x, and abs(x) is an operator that returnsan absolute value of the real number x.

In Step S110, the expansion section 133 expands the normalized quantizedcoefficient qcoef of the component X obtained by the processing of StepS109, by using the extension bit precision dBD per extension bitprecision control group SG as represented by the following Equation(12), and obtains the expanded quantized coefficient qcoefS′.At dBD>0,acoefS′=qcoef<<dBDAt dBD=0,qcoefS′=qcoef  (12)

In Step S111, the inverse quantization section 117 inversely quantizesthe quantized coefficient qcoefS′ obtained by the processing of StepS110 with characteristics corresponding to characteristics ofquantization of Step S108 and derives the transform coefficient coefS′.It is noted that description to be given (below) with respect to thedecoding side can be applied to the inverse quantization processing ofthis Step S111 since this inverse quantization processing is similar toinverse quantization processing performed on the decoding side (to bedescribed below).

In Step S112, the inverse orthogonal transform section 118 performsinverse orthogonal transform on the transform coefficient coefS'obtained by the processing of Step S111, by a method corresponding tothe orthogonal transform processing of Step S107, and derives theprediction residual resiS′. It is noted that description to be given(below) with respect to the decoding side can be applied to the inverseorthogonal transform processing of this Step S112 since this inverseorthogonal transform processing is similar to inverse orthogonaltransform processing performed on the decoding side (to be describedbelow).

In Step S113, the normalization section 134 normalizes the predictionresidual resiS′ of the component X obtained by the processing of StepS112, by using the extension bit precision dBD per extension bitprecision control group SG as represented by the following Equation(13), and obtains the normalized prediction residual resi′.At dBD>0, resi′=(resiS′+offsetBD)>>dBDwhere offsetBD=1<<(dBD−1)At dBD=0, resi′=resiS′  (13)

It is noted that normalization may be performed by the followingEquation (14) as an alternative to this Equation (13).At dBD>0, resi′=sign(resiS′)*(abs(resiS′)+offsetBD)>>dBDAt dBD=0, resi′=resiS′  (14)

In the Equation, sign(x) is an operator that returns a positive ornegative sign of a real number x, and abs(x) is an operator that returnsan absolute value of the real number x. It is noted that in the presentspecification, an index ‘as in “variable’” is intended to identify a‘variable’ corresponding to ‘variable’ on an encoding processing sideand obtained by inverse processing on a decoding processing side. Inaddition, index S as in “variableS” indicates an expanded ‘variable.’

In Step S114, the computing section 119 generates a decoded image thatis locally decoded, by adding the predicted image obtained by theprediction processing of Step S113 to the prediction residual resi′derived by the processing of Step S109.

In Step S115, the in-loop filter section 120 performs in-loop filterprocessing on the decoded image that is locally decoded and that isderived by the processing of Step S114.

In Step S116, the frame memory 121 stores the decoded image that islocally decoded and that is derived by the processing of Step S114 andthe decoded image that is locally decoded and that is filter-processedin Step S115.

In Step S117, the encoding section 115 encodes the normalized quantizedcoefficient qcoef obtained by the processing of Step S109. For example,the encoding section 115 encodes the quantized coefficient qcoef that isinformation associated with the image, by the arithmetic encoding or thelike, and generates encoded data. In addition, the encoding section 115encodes the various encoding parameters (header information Hinfo,prediction mode information Pinfo, transform information Tinfo, and thelike) at this time. Furthermore, the encoding section 115 derives theresidual information RInfo from the quantized coefficient qcoef andencodes the residual information RInfo.

In Step S118, the accumulation buffer 116 accumulates the encoded dataobtained in such way and outputs the encoded data, as, for example, abit stream, to outside of the image encoding apparatus 100. This bitstream is transmitted to the decoding side via, for example, thetransmission line and the recording medium. Moreover, the rate controlsection 123 exercises rate control as needed.

When processing of Step S118 is ended, the image encoding processing isended.

In the image encoding processing in the flow described above, it ispossible to execute the orthogonal transform processing and thequantization processing with improved computing precision, by expandingthe number of significant figures of the prediction residual resi thatis yet to be subjected to orthogonal transform, with use of theextension bit precision dBD in Step S106, and by normalizing thequantized coefficient qcoefS subjected to quantization, with use of theextension bit precision dBD in Step S109. In other words, encodingefficiency is improved by the improved computing precision.

Likewise, it is possible to execute the inverse quantization processingand the inverse orthogonal transform processing with improved computingprecision, by expanding the number of significant figures of thequantized coefficient qcoef that is yet to be subjected to inversequantization, with use of the extension bit precision dBD in Step S110,and by normalizing the prediction residual resiS′ subjected to inverseorthogonal transform, with use of the extension bit precision dBD inStep S113. In other words, encoding efficiency is improved by theimproved computing precision.

<Flow of dBD and deltaX Derivation Processing>

In Step S103 of FIG. 17, the various encoding parameters are derived.For example, the extension bit precision dBD, the difference parameterdeltaX, and the like are derived. An example of a flow of dBD and deltaXderivation processing for deriving such parameters as the extension bitprecision dBD and the difference parameter deltaX will be described withreference to a flowchart of FIG. 18.

When the dBD and deltaX derivation processing is started, the sequencebit depth setting section 151 of the control section 101 sets thesequence bit depth channelBitDepth of a component X on the basis ofexternal parameters in Step S131. In other words, the sequence bit depthsetting section 151 obtains a bit depth of each component of the inputimage input from outside and sets a value of the obtained bit depth tothe sequence bit depth channelBitDepth of each component.

In Step S152, the slice bit depth setting section 132 derives the slicebit depth sliceBitDepth of the component X.

For example, the slice bit depth setting section 152 first derives thefollowing syntaxes associated with the slice bit depth of each componentper slice, on the basis of a pixel group of each component contained inthe slice.

slice_min_val: minimum pixel value of component X (X=Y/Cb/Cr) withinslice

slice_max_val: maximum pixel value of component X (X=Y/Cb/Cr) withinslice

The slice bit depth setting section 152 then derives the slice bit depthsliceBitDepth by using those syntaxes, as represented by the followingEquation (15).sliceBitDepth=ceil(log 2(slice_max_val−slice_min_val))  (15)

In Step S133, the pixel minimum value/maximum value search section 153derives the minimum pixel value minPixelValu and the maximum pixel valuemaxPixelValue of the component X per extension bit precision controlgroup SG.

Here, the extension bit precision control group SG refers to a groupunit controlling the extension bit precision dBD for expanding thenumber of significant figures of the coefficient of the component X. Forexample, in a case of an input image 191 depicted in FIG. 19, an overallpicture is segmented into 4×4=16, and the segmented parts are set aslocal areas (local areas 00 to 33). For example, such a local area canbe designated as the extension bit precision control group SG. In otherwords, in this case, the extension bit precision dBD is derived perlocal area. In other words, the number of significant figures of thecoefficient (computing precision) is controlled per local area. Amagnitude and a shape of the area designated as such an extension bitprecision control group SG can be selected freely. For example, theextension bit precision control group SG may be set using, as a unit, aSlice, a Tile, a CTU (Coding Tree Unit), a CTB (Coded Tree Block), a CU(Coding Unit), a CB (Coding Block), a TU (Transform Unit), a TB(Transform Block), or the like.

In Step S134, the dBD derivation section 154 derives the local-level bitdepth LocalBitDepth (bit depth of each extension bit precision controlgroup SG) per extension bit precision control group SG by using theminimum pixel value minPixelValue and the maximum pixel valuemaxPixelValue of the component X derived in Step S133, as represented bythe following Equation (16). Furthermore, the dBD derivation section 154derives the extension bit precision dBD indicating by what bits thenumber of significant figures of the coefficient is to be expanded, byusing the local-level bit depth LocalBitDepth and the sequence bit depthchannelBitDepth derived in Step S131, as represented by, for example,the following Equation (17).

It is noted that the local bit depth LocalBitDepth derived in Equation(16) may further be corrected to LocalBitDepth=max(minLocalBitDepth,LocalBitDepth) by reference to the predetermined thresholdminLocalBitDepth (for example, eight), as represented in Equation (16A).By doing so, it is possible to suppress occurrence of the case in whichthe value of the extension bit precision dBD derived in subsequentEquation (17) becomes excessively large in value (that is, the number ofsignificant figures is excessively expanded, and intermediatecoefficient data exceeds the range from the minimum value coefMin to themaximum value coefMax). It is noted that in Equation (16A), the operatormax (x, y) is the operator that returns a larger value out of numericvalues x and y.LocalBitDepth=ceil(log 2(maxPixelValue−minPixelValue))  (16)LocalBitDepth=max(minLocalBitDepth,LocalBitDepth)   (16A)dBD=channelBitDepth−LocalBitDepth  (17)

In Step S135, the deltaX derivation section 155 derives the differenceparameter deltaX of the extension bit precision dBD of the component Xper extension bit precision control group SG. For example, the deltaXderivation section 155 derives the difference parameter by using thesequence bit depth channelBitDepth, the slice bit depth sliceBitDepth,and the extension bit precision dBD of each extension bit precisioncontrol group SG, as represented by the following Equation (18). Inother words, in this case, the difference parameter deltaX is adifference between the slice bit depth sliceBitDepth and the local-levelbit depth LocalBitDepth of the image.

$\begin{matrix}\begin{matrix}{{deltaX} = {{dBD} + {sliceBitDepth} - {channelBitDepth}}} \\{= {{sliceBitDepth} - {LocalBitDepth}}}\end{matrix} & (18)\end{matrix}$

It is noted that there is a relation ofchannelBitDepth≥sliceBitDepth≥deltaX. In such way, representing thedifference parameter deltaX via the extension bit precision dBD, thesequence bit depth, and the slice bit depth makes it possible to makesmaller (suppress an increase in) the value of the difference parameterdeltaX. In other words, it is possible to decrease the code amount ofthe deltaX and suppress a reduction in encoding efficiency.

When the processing of Step S135 is ended, then the dBD and deltaXderivation processing is ended, and the processing returns to FIG. 17.

<Control Over Extension Bit Precision Control Group SG>

It is noted that the extension bit precision control group SG may becontrolled by a difference value diff_cu_delta_depth of a segmentationdepth with respect to a size Ctb Log 2Size of a CTB. In this case, asize Log 2SGSize of the SG is obtained as represented by, for example,the following Equation (19).Log 2SGSize=Ctb Log 2Size−diff_cu_delta_depth  (19)

In a case of assuming, for example, that a CTB indicated by adotted-line frame in FIG. 20 has a size of 128×128 (Ctb Log 2Size=7) andthe SG indicated by a thick-line frame in FIG. 20 has a size of 32×32(Log 2SGSize=5), the difference value diff_cu_delta_depth is two asrepresented by the following Equation (20).diff_cu_delta_depth=Ctb Log 2Size−Log 2SGSize=7−5=2  (20)

It is noted that the difference value diff_cu_delta_depth is assumed tobe notified in units of predetermined parameter sets (headerinformation) (for example, a sequence parameter set (SPS), a pictureparameter set (PPS)), or by a slice header (SH) or the like). By doingso, it is possible to control granularity of the extension bit precisioncontrol group SG by the difference value diff_cu_delta_depth.

In a case, for example, in which a signal range is constant in a largearea to some extent, decreasing this difference valuediff_cu_delta_depth makes it possible to expand the unit of notificationof the difference parameter deltaX and to decrease overhead.

Conversely, in a case in which the signal range changes in a small area,increasing this difference value diff_cu_delta_depth makes it possibleto diminish the unit of notification of the difference parameter deltaXand to expand the number of significant figures of the transformcoefficient at the time of inverse quantization/inverse orthogonaltransform with finer granularity.

Other Example 1 of Definition of Difference Parameter deltaX

While it is described above that the difference parameter deltaX of theextension bit precision dBD of the component X is obtained from thesequence bit depth channelBitDepth, the slice bit depth sliceBitDepth,and the extension bit precision dBD, definition of the differenceparameter deltaX can be made freely and is not limited to this example.

For example, the extension bit precision dBD may be derived as a sum ofthe sequence bit depth channelBitDepth and the difference parameterdeltaX. In other words, the difference parameter deltaX may bedetermined as a difference value between the sequence bit depthchannelBitDepth and the extension bit precision dBD, as represented bythe following Equation (21)deltaX=channelBitDepth−dBD  (21)

By doing so, the difference parameter deltaX can be derived more easilythan the example described above.

Other Example 2 of Definition of Difference Parameter deltaX

Furthermore, the difference parameter deltaX may be defined to beequivalent to the extension bit precision dBD, as represented by thefollowing Equation (22).deltaX=dBD  (22)

In this case, the difference parameter deltaX can be derived withoutdependence on parameters other than the extension bit precision dBD.

Other Example 3 of Definition of Difference Parameter deltaX

Further, as represented by, for example, the following Equation (23),the difference parameter deltaX may be defined as a difference betweenthe extension bit precision dBD of the extension bit precision controlgroup SG to be processed (referred to as a “current extension bitprecision control group SG_(cur)”) and a predicted value dBD_(pred) ofthe extension bit precision dBD.deltaX=dBD−dBD _(pred)  (23)

For example, this predicted value dBD_(pred) is derived by referring todBDs of encoded extension bit precision control groups neighborSGs inthe neighborhood of the current extension bit precision control groupSG_(cur). In a CTB as depicted in A of FIG. 21, for example, it isassumed that a lower right quarter area is the current extension bitprecision control group SG_(cur) as depicted in B of FIG. 21. It is alsoassumed that the extension bit precision control group SG in theneighborhood of an upper side of the current extension bit precisioncontrol group SG_(cur) is a neighboring extension bit precision controlgroup SG_(A) and the extension bit precision control group SG in theneighborhood of a left side of the current extension bit precisioncontrol group SG_(cur) is a neighboring extension bit precision controlgroup SG₃. The neighboring extension bit precision control groups SG_(A)and SG₃ are encoded neighboring extension bit precision control groups.

In this case, the predicted value dBD_(pred) of the extension bitprecision dBD may be set on the basis of extension bit precision dBD_(A)of the neighboring extension bit precision control group SG_(A) andextension bit precision dBD_(B) of the neighboring extension bitprecision control group SG₃. Furthermore, the predicted value dBD_(pred)of the extension bit precision dBD may be derived, for example, by amethod according to whether or not these neighboring extension bitprecision control groups can be referred to. For example, the predictedvalue dBD_(pred) may be obtained by the following method.

In a case, for example, in which both the neighboring extension bitprecision control groups SG_(A) and SG₃ can be referred to, thepredicted value dBD_(pred) is derived as represented by the followingEquation (24).dBD _(pred) =ave(dBD _(A) ,dBD _(B))  (24)

In Equation (18), ave(x, y) is an operator that returns an average valueof x and y.

For example, ave(x, y)=(x+y+1)>>1.

Moreover, in a case, for example, in which only the neighboringextension bit precision control group SG_(A) can be referred to, thepredicted value dBD_(pred) is derived as represented by the followingEquation (25).dBD _(pred) =dBD _(A)  (25)

Further, in a case, for example, in which only the neighboring extensionbit precision control group SG₃ can be referred to, the predicted valuedBD_(pred) is derived as represented by the following Equation (26).dBD _(pred) =dBD _(B)  (26)

Further, in a case, for example, in which neither the neighboringextension bit precision control group SG_(A) nor the neighboringextension bit precision control group SG₃ can be referred to, thepredicted value dBD_(pred) is derived as represented by the followingEquation (27).dBD _(pred)=const(=0)  (27)

In such way, predicting the dBD of the current extension bit precisioncontrol group SG_(cur) by using each of the extension bit precision dBDof the encoded neighboring extension bit precision control groupsneighborSGs makes it possible to decrease the value of the differenceparameter deltaX. In other words, it is possible to suppress a growth ofthe code amount required for decoding or encoding the deltaX.

<Flow of Extension Bit Precision Information Encoding Processing>

In the encoding processing of Step S117 of FIG. 17, the information,such as the difference parameter deltaX of the extension bit precisiondBD, associated with the control over the number of significant figures(control over the computing precision) and supplied from the controlsection 101 is also encoded and contained in the bit stream.

An example of a flow of extension bit precision information encodingprocessing for encoding such information associated with the controlover the number of significant figures (control over the computingprecision) will be described with reference to a flowchart of FIG. 22.

When the extension bit precision information encoding processing isstarted, the encoding section 115 encodes information associated withthe sequence bit depth channelBitDepth of each component (syntaxvalues), as one parameter in the sequence parameter set in Step S151.Examples of syntaxes associated with channelBitDepth are as follows.

bit_depth_luma_minus8: syntax indicating sequence bit depth of luminance

bit_depth_chroma_minus8: syntax indicating sequence bit depth ofchrominance

These syntax values are derived as represented by, for example, thefollowing Equations (28) and (29). For example, the sequence bit depthsetting section 151 of the control section 101 derives these syntaxvalues in Step S131 of FIG. 18.bit_depth_luma_minus8=channelBitDepth−8  (28)bit_depth_chroma_minus8=channelBitDepth−8  (29)

The encoding section 115 encodes these syntax values supplied from thecontrol section 101, as the information associated with the sequence bitdepth channelBitDepth, and contains the encoded result in a bit stream(generates a bit stream containing the information associated with thesequence bit depth).

In Step S152, the encoding section 115 encodes information associatedwith the slice bit depth sliceBitDepth of each component (syntaxvalues), as one parameter in the slice header SH. Examples of syntaxesassociated with the slice bit depth sliceBitDepth are as follows.

slice_min_val: minimum pixel value of component X (X=Y/Cb/Cr) withinslice

slice_max_val: maximum pixel value of component X (X=Y/Cb/Cr) withinslice

For example, the slice bit depth setting section 152 of the controlsection 101 derives these syntax values in Step S132 of FIG. 18. Theencoding section 115 encodes these syntax values supplied from thecontrol section 101, as the information associated with the slice bitdepth sliceBitDepth, and contains the encoded result in a bit stream(generates a bit stream containing the information associated with theslice bit depth).

In Step S153, the encoding section 115 encodes the difference parameterdeltaX of the extension bit precision dBD of the component X perextension bit precision control group SG. For example, the deltaXderivation section 155 of the control section 101 derives thisdifference parameter deltaX in Step S135 of FIG. 18. The encodingsection 115 encodes the difference parameter deltaX supplied from thecontrol section 101, as the information associated with the extensionbit precision dBD, and contains the encoded difference parameter deltaXin a bit stream (generates a bit stream containing the differenceparameter deltaX).

When processing of Step S153 is ended, then the extension bit precisioninformation encoding processing is ended, and the processing returns toFIG. 17.

As described above, by encoding the information associated with thecontrol over the number of significant figures such as the differenceparameter deltaX (control over the computing precision), containing theencoded information in the bit stream, and transmitting the bit streamfrom the encoding side to the decoding side, it is possible for thedecoding side to perform decoding by the method to which the presenttechnology is applied. It is therefore possible to suppress a reductionin encoding efficiency.

FIG. 23 is a diagram depicting an example of syntaxes of the differenceparameter deltaX. The syntax in A of FIG. 23 depicts an example of acase of signaling the difference parameter deltaX in larger units thanthe transform blocks such as CTUs, CTBs, CUs, or CBs. This can decreaseoverhead.

It is noted that adaptive_scaling_enabled_flag is a flag indicatingwhether or not adaptive scaling is applicable, andadaptive_scaling_enabled_flag is notified by header information (forexample, the sequence parameter set (SPS), the picture parameter set(PPS), or the slice header (SH)).

When the flag is true (1), the adaptive scaling is applied; thus, asyntax group associated with adaptive scaling is decoded/encoded. On theother hand, when the flag is false (0), the adaptive scaling is notapplied; thus, it is interpreted that deltaX(X=Y/Cb/Cr)=0.

The syntax in B of FIG. 23 indicates an example of a case of signalingthe different parameter deltaX in transform block units. In this case,the difference parameter is encoded/decoded with respect to thetransform block having a significant coefficient, by referring to a cbf(coded block flag) of each component. This can decrease overhead.

<Image Decoding Apparatus>

Next, configurations of controlling the computing precision in themethod #1 in the table of FIG. 9 at the time of decoding will bedescribed. FIG. 24 is a block diagram depicting an example ofconfigurations of an image decoding apparatus that is one aspect of theimage processing apparatus to which the present technology is applied.An image decoding apparatus 200 depicted in FIG. 24 is an apparatus thatdecodes encoded data obtained by encoding a prediction residual betweenan image and a predicted image thereof such as an AVC or HEVC-compliantapparatus. For example, the image decoding apparatus 200 implements thetechnologies described in NPL 1 to NPL 3 and decodes encoded dataobtained by encoding image data regarding an image sequence with use ofa method compliant with standards described in any of those documents.For example, the image decoding apparatus 200 decodes encoded data (bitstream) generated by the image encoding apparatus 100 described above.

It is noted that principal configurations such as processing sectionsand flows of data depicted in FIG. 24 are not necessarily allconfigurations. In other words, processing sections that are notdepicted as blocks in FIG. 24 may be present or processing and flows ofdata that are not indicated by arrows or the like in FIG. 24 may bepresent in the image decoding apparatus 200. This applies to otherdrawings for illustrating processing sections and the like within theimage decoding apparatus 200.

In FIG. 24, the image decoding apparatus 200 has an accumulation buffer211, a decoding section 212, an inverse quantization section 213, aninverse orthogonal transform section 214, a computing section 215, anin-loop filter section 216, a reordering buffer 217, a frame memory 218,and a prediction section 219. It is noted that the prediction section219 has an intra prediction section and an inter prediction section thatare not depicted. The image decoding apparatus 200 is an apparatus forgenerating image sequence data by decoding the encoded data (bitstream).

It is noted that the image decoding apparatus 200 further has anexpansion section 231 and a normalization section 232.

<Accumulation Buffer>

The accumulation buffer 211 acquires the bit stream input to the imagedecoding apparatus 200 and holds (stores) the bit stream. Theaccumulation buffer 211 supplies the accumulated bit stream to thedecoding section 212 either at a predetermined timing or in a case, forexample, in which a predetermined condition is satisfied.

<Decoding Section>

The decoding section 212 performs processing associated with imagedecoding. For example, the bit stream supplied from the accumulationbuffer 211 is input to the decoding section 212, and the decodingsection 212 performs variable-length decoding on a syntax value of eachsyntax element from a bit sequence thereof according to a definition ofa syntax table and derives parameters.

The parameters derived from the syntax elements and the syntax values ofthe syntax elements contain information such as the header informationHinfo, the prediction mode information Pinfo, the transform informationTinfo, the residual information RInfo, and the filter information Finfo,for example. In other words, the decoding section 212 parses thesepieces of information (analyzes and acquires these pieces ofinformation) from the bit stream. These pieces of information willhereinafter be described.

<Header Information Hinfo>

The header information Hinfo contains header information which is, forexample, a VPS (Video Parameter Set), an SPS (Sequence Parameter Set), aPPS (Picture Parameter Set), and an SH (slice header). The headerinformation Hinfo contains information specifying, for example, an imagesize (width PicWidth, height PicHeight), bit depths (luminanncebitDepthY, chrominance bitDepthC), a chrominance array typeChromaArrayType, a maximum value MaxCUSize/minimum value MinCUSize of aCU size, a maximum depth MaxQTDepth/minimum depth MinQTDepth of aquadtree segmentation (also referred to as a “Quad-tree segmentation”),maximum depth MaxBTDepth/minimum depth MinBTDepth of Binary-treesegmentation, a maximum value MaxTSSize of a transform skip block (alsoreferred to as a “maximum transform skip block size”), and an on-offflag (also referred to as an “enabled flag”) of each encoding tool.

Examples of the on-off flag of each encoding tool contained in theheader information Hinfo include an on-off flag associated withtransform and quantization processing described below. It is noted thatthe on-off flag of the encoding tool can also be interpreted as a flagthat indicates whether or not a syntax associated with the encoding toolis present in the encoded data. In addition, a case in which a value ofthe on-off flag is 1 (true) indicates that the encoding tool isavailable, and a case in which the value of the on-off flag is 0 (false)indicates that the encoding tool is unavailable. It is noted that theflag values can be interpreted reversely.

Cross-component prediction enabled flag (ccp_enabled_flag): flaginformation indicating whether or not cross-component prediction (CCP)(also referred to as “CC prediction”) is available. For example, a casein which this flag information is “1” (true) indicates that the CCP isavailable, and a case in which the flag information is “0” (false)indicates that the CCP is unavailable.

It is noted that this CCP is also referred to as a “cross-componentlinear prediction (CCLM or CCLMP).”

<Prediction Mode Information Pinfo>

The prediction mode information Pinfo contains, for example, informationsuch as size information PBSize (prediction block size), intraprediction mode information IPinfo, and movement prediction informationMVinfo regarding a PB (prediction block) to be processed.

The intra prediction mode information IPinfo contains, for example,prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode inJCTVC-W1005, 7.3.8.5 Coding Unit syntax and a luminance intra predictionmode IntraPredModeY derived from the syntax.

Further, the intra prediction mode information IPinfo contains, forexample, a cross-component prediction flag (ccp_flag (cclmp_flag)), amulti-class linear prediction mode flag (mclm_flag), a chrominancesample location type identifier (chroma_sample_loc_type_idx), achrominance MPM identifier (chroma_mpm_idx), and a luminance intraprediction mode (IntraPredModeC) derived from these syntaxes.

The cross-component prediction flag (ccp_flag (cclmp_flag)) is flaginformation indicating whether or not to apply the cross-componentlinear prediction. For example, ccp_flag==1 indicates that thecross-component prediction is to be applied, and ccp_flag==0 indicatesthat the cross-component prediction is not to be applied.

The multi-class linear prediction mode flag (mclm_flag) is informationassociated with a linear prediction mode (linear prediction modeinformation). More specifically, the multi-class linear prediction modeflag (mclm_flag) is flag information indicating whether or not to set amulti-class linear prediction mode. For example, a case of “0” indicatesa one-class mode (single-class mode) (for example, CCLMP), and a case of“1” indicates a two-class mode (multi-class mode) (for example, MCLMP).

The chrominance sample location type identifier(chroma_sample_loc_type_idx) is an identifier that identifies a type ofa pixel location of a chrominance component (also referred to as a“chrominance sample location type”). For example, in a case in which thechrominance array type (ChromaArrayType) that is information associatedwith a color format indicates format 420, the chrominance samplelocation type identifier is assigned as represented by the followingExpression (30).chroma_sample_loc_type_idx==0:Type2chroma_sample_loc_type_idx==1 Type3chroma_sample_loc_type_idx==2:Type1chroma_sample_loc_type_idx==3:Type0  (30)

It is noted that this chrominance sample location type identifier(chroma_sample_loc_type_idx) is transmitted as information associatedwith pixel locations of a chrominance component (chroma_sample_loc_info()) (transmitted while being stored in the information associated withpixel locations of a chrominance component (chroma_sample_loc_info( )).

The chrominance MPM identifier (chroma_mpm_idx) is an identifierrepresenting which prediction mode candidate in a chrominance intraprediction mode candidate list (intraPredModeCandListC) is to bedesignated as the chrominance intra prediction mode.

The movement prediction information MVinfo contains, for example,information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX,mvp_lX_flag, X={0,1}, and mvd (refer to, for example, JCTVC-W1005,7.3.8.6 Prediction Unit Syntax).

Needless to say, the information to be contained in the prediction modeinformation Pinfo may be any information, and information other thanthose pieces of information may be contained in the prediction modeinformation Pinfo.

<Transform Information Tinfo>

The transform information Tinfo contains, for example, the followingpieces of information. Needless to say, the information to be containedin the transform information Tinfo may be any information, andinformation other than those pieces of information may be contained inthe transform information Tinfo.

A width size TBWSize and a height size TBHSize of the transform block tobe processed (which may be base 2 logarithmic values log 2TBWSize (orlog 2TrWidth) and log 2TBHSize (or log 2TrHeight) of TBWSize (orTrWidth) and TBHSize (or TrHeight)).

A transform skip flag (ts_flag): a flag indicating whether or not toskip (inverse) primary transform and (inverse) secondary transform.

Scan identifier (scanIdx)

Quantization parameter (qp)

Quantization matrix (scaling_matrix (for example, JCTVC-W1005, 7.3.4Scaling list data syntax))

<Residual Information RInfo>

The residual information RInfo (refer to, for example, 7.3.8.11 ResidualCoding syntax of JCTVC-W005) contains, for example, the followingsyntaxes.

cbf (coded_block_flag): a flag indicating whether or not residual datais present

last_sig_coeff_x_pos: a last nonzero coefficient X coordinate

last_sig_coeff_y_pos: a last nonzero coefficient Y coordinate

coded_sub_block_flag: a flag indicating whether or not a sub-blocknonzero coefficient is present

sig_coeff_flag: a flag indicating whether or not a nonzero coefficientis present

gr1_flag: a flag indicating whether a level of a nonzero coefficient isgreater than one (also referred to as a “GR1_flag”)

gr2_flag: a flag indicating whether a level of a nonzero coefficient isgreater than two (also referred to as a “GR2_flag”)

sign_flag: a sign indicating whether a nonzero coefficient is positiveor negative (also referred to as a “sign code”)

coeff_abs_level_remaining: remaining level of a nonzero coefficient(also referred to as a “nonzero coefficient remaining level)

Needless to say, the information to be contained in the residualinformation RInfo may be any information, and information other thanthese pieces of information may be contained in the residual informationRInfo.

<Filter Information Finfo>

The filter information Finfo contains, for example, control informationassociated with each of the following filter processing.

Control information associated with the deblocking filter (DBF)

Control information associated with the pixel adaptive offset (SAO)

Control information associated with the adaptive loop filter (ALF)

Control information associated with other linear/nonlinear filters

More specifically, the filter information Finfo contains, for example,information designating a picture or an area in the picture to whicheach filter is applied, filter On/Off control information in each CU,and filter On/Off control information associated with boundaries ofslices and tiles. Needless to say, the information to be contained inthe filter information Finfo may be any information, and informationother than these pieces of information may be contained in the filterinformation Finfo.

The decoding section 212 will be described again. The decoding section212 refers to the residual information RInfo and derives the quantizedcoefficient qcoef at each coefficient location within each transformblock. The decoding section 212 supplies the quantized coefficient qcoefto the expansion section 231.

Further, the decoding section 212 supplies the header information Hinfo,the prediction mode information Pinfo, the quantized coefficient qcoef,the transform information Tinfo, and the filter information Finfo thatare completed with parsing to blocks. Specifically, the decoding section212 supplies the parsed information as follows.

The header information Hinfo is supplied to the inverse quantizationsection 213, the inverse orthogonal transform section 214, theprediction section 219, and the in-loop filter section 216.

The prediction mode information Pinfo is supplied to the inversequantization section 213 and the prediction section 219.

The transform information Tinfo is supplied to the inverse quantizationsection 213, the inverse orthogonal transform section 214, the expansionsection 231, and the normalization section 232.

The filter information Finfo is supplied to the in-loop filter section216.

Needless to say, supply destinations described above are one example andare not limited to this example. For example, each encoding parametermay be supplied to a freely-selected processing section. Furthermore,the other pieces of information may be supplied to a freely-selectedprocessing section.

For example, the information associated with the control over theeffective coefficient (control over computing precision) such as theinformation associated with the sequence bit depth channelBitDepth, theinformation associated with the slice bit depth sliceBitDepth, and thedifference parameter deltaX described above in the description of theimage encoding apparatus 100 may be contained in the transforminformation Tinfo.

<Inverse Quantization Section>

The inverse quantization section 213 performs processing associated withinverse quantization. For example, the transform information Tinfo andthe quantized coefficient qcoefS′ supplied from the expansion section231 are input to the inverse quantization section 213, and the inversequantization section 213 scales (inversely quantizes) the value of thequantized coefficient qcoefS′ on the basis of the transform informationTinfo and derives the transform coefficient coefS' subjected to theinverse quantization.

It is noted that this inverse quantization is performed as inverseprocessing of the quantization performed by the quantization section114. In addition, this inverse quantization is processing similar to theinverse quantization performed by the inverse quantization section 117.In other words, the inverse quantization section 117 performs theprocessing (inverse quantization) similar to that performed by theinverse quantization section 213.

The inverse quantization section 213 supplies the derived transformcoefficient coefS' to the inverse orthogonal transform section 214.

<Inverse Orthogonal Transform Section>

The inverse orthogonal transform section 214 performs processingassociated with inverse orthogonal transform. For example, the transformcoefficient coefS' supplied from the inverse quantization section 213and the transform information Tinfo supplied from the decoding section212 are input to the inverse orthogonal transform section 214, and theinverse orthogonal transform section 214 performs inverse orthogonaltransform processing on the transform coefficient coefS' on the basis ofthe transform information Tinfo and derives the prediction residualresiS′.

It is noted that this inverse orthogonal transform is performed asinverse processing of the orthogonal transform performed by theorthogonal transform section 113. In addition, this inverse orthogonaltransform is processing similar to the inverse orthogonal transformperformed by the inverse orthogonal transform section 118. In otherwords, the inverse orthogonal transform section 118 performs theprocessing (inverse orthogonal transform) similar to that performed bythe inverse orthogonal transform section 214.

The inverse orthogonal transform section 214 supplies the derivedprediction residual resiS′ to the normalization section 232.

<Computing Section>

The computing section 215 performs processing associated with additionof information associated with the image. For example, the predictionresidual resi′ supplied from the normalization section 232 and thepredicted image P supplied from the prediction section 219 are input tothe computing section 215. The computing section 215 adds up theprediction residual resi′ and the predicted image P (predicted signal)corresponding to the prediction residual resi′ and derives the localdecoded image R_(local), as represented by the following Equation (31).R _(local)=resi′+P  (31)

The computing section 215 supplies the derived local decoded imageR_(local) to the in-loop filter section 216 and the frame memory 218.

<In-Loop Filter Section>

The in-loop filter section 216 performs processing associated within-loop filter processing. For example, the local decoded imageR_(local) supplied from the computing section 215 and the filterinformation Finfo supplied from the decoding section 212 are input tothe in-loop filter section 216. It is noted that information to be inputto the in-loop filter section 216 may be any information, andinformation other than these pieces of information may be input to thein-loop filter section 216.

The in-loop filter section 216 performs filter processing on the localdecoded image R_(local) on the basis of the filter information Finfo asappropriate.

For example, as described in NPL 3, the in-loop filter section 216applies four in-loop filters, which are the bilateral filter, thedeblocking filter (DBF (DeBlocking Filter)), the adaptive offset filter(SAO (Sample Adaptive Offset), and the adaptive loop filter (ALF(Adaptive Loop Filter)), in this order. It is noted that the specificfilters and order of application thereof can be selected freely, andselection can be made as appropriate.

The in-loop filter section 216 performs filter processing correspondingto the filter processing performed by the encoding side (for example,in-loop filter section 120 of the image encoding apparatus 100).Needless to say, the filter processing to be performed by the in-loopfilter section 216 can be selected freely and is not limited to anexample described above. For example, the in-loop filter section 216 mayapply a Wiener filter and the like.

The in-loop filter section 216 supplies the filter-processed localdecoded image R_(local) to the reordering buffer 217 and the framememory 218.

<Reordering Buffer>

The local decoded image R_(local) supplied from the in-loop filtersection 216 is input to the reordering buffer 217, and the reorderingbuffer 217 holds (stores) the local decoded image R_(local) Thereordering buffer 217 reconstructs the decoded image R of each pictureunit by using the local decoded image R_(local) and holds the decodedimage R (stores the decoded image R in a buffer). The reordering buffer217 reorders the obtained decoded images R from an order of decoding toan order of reproduction. The reordering buffer 217 outputs thereordered decoded image R group to outside of the image decodingapparatus 200, as image sequence data.

<Frame Memory>

The frame memory 218 performs processing related to storage of dataassociated with images. For example, the local decoded image R_(local)supplied by the computing section 215 is input to the frame memory 218,and the frame memory 218 reconstructs the decoded image R of eachpicture unit and stores the decoded image R in a buffer of the framememory 218.

Further, the in-loop-filter-processed local decoded image R_(local)supplied from the in-loop filter section 216 is input to the framememory 218, and the frame memory 218 reconstructs the decoded image R ofeach picture unit and stores the decoded image R in the buffer of theframe memory 218. The frame memory 218 supplies the stored decoded imageR (or part of the decoded image R) to the prediction section 219, as areference image, as appropriate.

It is noted that the frame memory 218 may store the header informationHinfo, the prediction mode information Pinfo, the transform informationTinfo, the filter information Finfo, and the like related to generationof the decoded image.

<Prediction Section>

The prediction section 219 performs processing associated withgeneration of a predicted image. For example, the prediction modeinformation Pinfo supplied from the decoding section 212 is input to theprediction section 219, and the prediction section 219 performsprediction by a prediction method designated by the prediction modeinformation Pinfo and derives the predicted image P. At the time of thederivation, the prediction section 219 uses either the pre-filteringprocessing or post-filtering processing decoded image R (or part of thedecoded image R) designated by the prediction mode information Pinfo andstored in the frame memory 218, as a reference image. The predictionsection 219 supplies the derived predicted image P to the computingsection 215.

<Expansion Section>

The quantized coefficient qcoef supplied from the decoding section 212and the transform information Tinfo supplied from the control section101 are input to the expansion section 231, and the expansion section231 performs processing associated with control over (expansion of) thenumber of significant figures of the quantized coefficient qcoef. Forexample, the expansion section 231 bit-shifts the quantized coefficientqcoef to the left by extension bit precision dBD bits (<<dBD) andexpands the number of significant figures of the quantized coefficientqcoef. In other words, the expansion section 231 expands the number ofsignificant figures of the quantized coefficient qcoef that is yet to besubjected to orthogonal transform and quantization, according to thechange of the bit depth in the time direction or the spatial directionwithin each sequence. Further, the expansion section 231 supplies thequantized coefficient qcoefS′ the number of significant figures of whichis expanded and which is obtained by the bit-shifting, to the inversequantization section 213.

<Normalization Section>

The prediction residual resiS′ supplied from the inverse orthogonaltransform section 214 and the transform information Tinfo supplied fromthe control section 101 are input to the normalization section 232, andthe normalization section 232 performs processing associated withcontrol over (normalization of) the number of significant figures of theprediction residual resiS′. For example, the normalization section 232bit-shifts the prediction residual resiS′ to the right by extension bitprecision dBD bits (>>dBD) and normalizes the number of significantfigures of the prediction residual resiS′. In other words, thenormalization section 232 normalizes the number of significant figuresof the prediction residual resiS′ subjected to inverse quantization andinverse orthogonal transform, according to the change of the bit depthin the time direction or the spatial direction within each sequence.Furthermore, the normalization section 232 supplies the predictionresidual resiS′ the number of significant figures of which is normalizedand which is obtained by the bit-shifting, to the computing section 215.

With the configurations described above, the image decoding apparatus200 can perform each of the inverse quantization processing and theinverse orthogonal transform processing in a state in which the numberof significant figures of each coefficient is further expanded fromexpansion of the sequence level and improve the computing precision ofthose series of processing. In other words, it is possible to suppress areduction in encoding efficiency.

<Details of Decoding Section>

FIG. 25 is a block diagram depicting an example of principalconfigurations of the decoding section 212, the principal configurationsbeing related to extraction of information associated with control overthe number of significant figures. As depicted in FIG. 25, the decodingsection 212 has a sequence bit depth derivation section 251, a slice bitdepth derivation section 252, a deltaX decoding section 253, and a dBDderivation section 254. While the decoding section 212 performsprocessing other than the extraction or the like of the informationassociated with the control over the number of significant figures,i.e., decoding of other pieces of information, as described above,description of configurations with respect to such series of processingis omitted.

The bit stream bitstream in the accumulation buffer 211 is input to thesequence bit depth derivation section 251, and the sequence bit depthderivation section 251 performs processing associated with derivation ofthe sequence bit depth channelBitDepth. For example, the sequence bitdepth derivation section 251 decodes and extracts information associatedwith the sequence bit depth contained in the bit stream. Further, forexample, the sequence bit depth derivation section 251 drives thesequence bit depth channelBitDepth by using the extracted informationassociated with the sequence bit depth. Moreover, for example, thesequence bit depth derivation section 251 supplies the derived sequencebit depth channelBitDepth to the dBD derivation section 254.Furthermore, for example, the sequence bit depth derivation section 251supplies the derived sequence bit depth channelBitDepth to the inversequantization section 213 and the inverse orthogonal transform section214, as the transform information Tinfo.

The bit stream bitstream in the accumulation buffer 211 is input to theslice bit depth derivation section 252, and the slice bit depthderivation section 252 performs processing associated with derivation ofthe slice bit depth sliceBitDepth. For example, the slice bit depthderivation section 252 decodes and extracts information associated withthe slice bit depth contained in the bit stream. Further, for example,the slice bit depth derivation section 252 derives the slice bit depthsliceBitDepth by using the derived information associated with the slicebit depth. Moreover, for example, the slice bit depth derivation section252 supplies the derived slice bit depth sliceBitDepth to the dBDderivation section 254.

The bit stream bitstream in the accumulation buffer 211 is input to thedeltaX decoding section 253, and the deltaX decoding section 253performs processing associated with decoding of the difference parameterdeltaX. For example, the deltaX decoding section 253 decodes andextracts the difference parameter deltaX (information associated withthe extension bit precision) contained in the bit stream. The deltaXdecoding section 253 supplies the obtained difference parameter deltaXto the dBD derivation section 254.

The sequence bit depth channelBitDepth supplied from the sequence bitdepth derivation section 251, the slice bit depth sliceBitDepth suppliedfrom the slice bit depth derivation section 252, and the differenceparameter deltaX supplied from the deltaX decoding section 253 are inputto the dBD derivation section 254, and the dBD derivation section 254performs processing associated with derivation of the extension bitprecision dBD. For example, the dBD derivation section 254 derives theextension bit precision dBD per local level on the basis of thoseparameters. In addition, the dBD derivation section 254 supplies thederived extension bit precision dBD to the inverse quantization section213, the inverse orthogonal transform section 214, the expansion section231, and the normalization section 232, as the transform informationTinfo.

<Flow of Image Decoding Processing>

A flow of each processing executed by the image decoding apparatus 200described above will next be described. An example of a flow of imagedecoding processing will first be described with reference to aflowchart of FIG. 26.

When the image decoding processing is started, the accumulation buffer211 acquires the encoded data (bit stream) supplied from outside of theimage decoding apparatus 200 and holds (accumulates) the encoded data inStep S201.

In Step S202, the decoding section 212 decodes the encoded data (bitstream) and obtains the quantized coefficient qcoef. In addition, thedecoding section 212 parses (analyzes and acquires) various encodingparameters from the encoded data (bit stream) by this decoding.

In Step S203, the expansion section 231 expands the quantizedcoefficient qcoef of the component X obtained by the processing of StepS202, by using the extension bit precision dBD per extension bitprecision control group SG as represented by the following Equation(32), and obtains the expanded quantized coefficient qcoefS′.At dBD>0,qcoefS′=qcoef<<dBDAt dBD=0,qcoefS′=qcoef  (32)

In Step S204, the inverse quantization section 213 performs inversequantization, which is the inverse processing of the quantizationperformed on the encoding side, on the quantized coefficient qcoefS′expanded by the processing of Step S203 and obtains the transformcoefficient coefS′.

In Step S205, the inverse orthogonal transform section 214 performsinverse orthogonal transform processing, which is the inverse processingof the orthogonal transform processing performed on the encoding side,on the transform coefficient coefS' obtained by the processing of StepS204 and obtains the prediction residual resi′.

In Step S206, the normalization section 232 normalizes the predictionresidual resiS′ of the component X obtained by the processing of StepS205, by using the extension bit precision dBD per extension bitprecision control group SG as represented by the following Equation(33), and obtains the normalized prediction residual resi.At dBD>0, resi′=(resiS′+offsetBD)>>dBDwhere offsetBD=1<<(dBD−1)At dBD=0, resi′=resiS′  (33)

Normalization may be performed by the following Equation (34) as analternative to this Equation (33).At dBD>0, resi′=sign(resiS′)*(abs(resiS′)+offsetBD)>>dBDAt dBD=0, resi′=resiS′  (34)

In the Equation, sign(x) is an operator that returns a positive ornegative sign of a real number x, and abs(x) is an operator that returnsan absolute value of the real number x.

In Step S207, the prediction section 219 executes prediction processingby a prediction method designated by the encoding side, on the basis ofthe information parsed in Step S202, and generates the predicted image Pby, for example, referring to the reference image stored in the framememory 218.

In Step S208, the computing section 215 adds up the prediction residualresi′ obtained by the processing of Step S206 and the predicted image Pobtained by the processing of Step S207 and derives the local decodedimage R_(local).

In Step S209, the in-loop filter section 216 performs in-loop filterprocessing on the local decoded image R_(local) obtained by theprocessing of Step S208.

In Step S210, the reordering buffer 217 derives the decoded image R byusing the filter-processed local decoded image R_(local) obtained by theprocessing of Step S307 and reorders the decoded image R group from theorder of decoding to the order of reproduction. The decoded image Rgroup reordered to the order of reproduction is output to outside of theimage decoding apparatus 200, as an image sequence.

Moreover, in Step S211, the frame memory 218 stores at least one of thelocal decoded image R_(local) obtained by the processing of Step 208 orthe filter-processed local decoded image R_(local) obtained by theprocessing of Step S209.

When the processing of Step S211 is ended, the image decoding processingis ended.

In the image decoding processing in the flow described above, it ispossible to execute the inverse quantization processing and the inverseorthogonal transform processing with the improved computing precision,by expanding the number of significant figures of the quantizedcoefficient qcoef that is yet to be subjected to inverse quantization,with use of the extension bit precision dBD in Step S203, and bynormalizing the prediction residual resiS′ subjected to inverseorthogonal transform, with use of the extension bit precision dBD inStep S206. In other words, encoding efficiency is improved by theimproved computing precision.

<Flow of dBD Derivation Processing>

In Step S202 of FIG. 26, various encoding parameters are derived. Forexample, the extension bit precision dBD and the like are derived. Anexample of a flow of the dBD derivation processing for deriving suchparameters as this extension bit precision dBD will be described withreference to a flowchart of FIG. 27.

When the dBD derivation processing is started, the sequence bit depthderivation section 251 of the decoding section 212 derives the sequencebit depth channelBitDepth of the component X in Step S231.

For example, the sequence bit depth derivation section 251 decodes andextracts syntax values associated with the sequence bit depthchannelBitDepth of each component (information associated with thesequence bit depth) contained in the sequence parameter set SPS of thebit stream. In addition, the sequence bit depth derivation section 251derives the sequence bit depth channelBtiDepth from the extracted syntaxvalues. Examples of syntaxes associated with channelBitDepth are asfollows.

bit_depth_luma_minus8: syntax indicating sequence bit depth of luminance

bit_depth_chroma_minus8: syntax indicating sequence bit depth ofchrominance

In a case of, for example, luminance, the sequence bit depth derivationsection 251 derives the sequence bit depth channelBtiDepth, asrepresented by the following Equation (35).channelBitDepth=8+bit_depth_luma_minus8  (35)

Furthermore, in a case of, for example, chrominance, the sequence bitdepth derivation section 251 derives the sequence bit depthchannelBitDepth, as represented by the following Equation (36).channelBitDepth=8+bit_depth_chroma_minus8  (36)

In Step S232, the slice bit depth derivation section 252 derives theslice bit depth sliceBitDepth of the component X.

For example, the slice bit depth derivation section 252 decodes andextracts syntax values associated with slice bit depth of each component(information associated with the slice bit depth) contained in the sliceheader SH of the bit stream. In addition, the slice bit depth derivationsection 252 derives the slice bit depth sliceBitDepth from the extractedsyntax values. Examples of syntaxes associated with the slice bit depthsliceBitDepth are as follows.

slice_min_val: minimum pixel value of component X (X=Y/Cb/Cr) withinslice

slice_max_val: maximum pixel value of component X (X=Y/Cb/Cr) withinslice

The slice bit depth derivation section 252 derives the slice bit depthsliceBitDepth from these values, as represented by, for example, thefollowing Equation (37).sliceBitDepth=ceil(log 2(slice_max_val−slice_min_val))  (37)

It is noted that the slice bit depth sliceBitDepth may be decodeddirectly from the slice header as an alternative to the minimum pixelvalue slice_min_val and the maximum pixel value slice_max_val withineach slice, and the difference value slice_bit_depth_diffX between thesequence bit depth channelBitDepth and the slice bit depth sliceBitDepthmay be decoded.

slice_bit_depth_diffX: a difference value of slice bit depth withrespect to sequence bit depth channelBitDepth of component X

In a case of deriving the slice bit depth sliceBitDepth fromslice_bit_depth_diffX, the slice bit depth sliceBitDepth is derived, asrepresented by, for example, the following Equation (38).sliceBitDepth=channelBitDepth−slice_bit_depth_diffX  (38)

In Step S233, the deltaX decoding section 253 decodes the differenceparameter deltaX of the extension bit precision dBD of the component Xper extension bit precision control group SG.

For example, the deltaX decoding section 253 decodes the differenceparameter deltaX of the extension bit precision dBD indicating by howmany bits the number of significant figures of the coefficient of thecomponent X is expanded per extension bit precision control group SG(Scaling Group), from the encoded data.

In Step S234, the dBD derivation section 254 derives the extension bitprecision dBD of the component X, per extension bit precision controlgroup SG.

For example, the dBD derivation section 254 derives the extension bitprecision dBD from the difference parameter deltaX of the extension bitprecision dBD of the component X, the sequence bit depthchannelBitDepth, and the slice bit depth sliceBitDepth, per extensionbit precision control group SG, as represented by the following Equation(39).dBD=channelBitDepth−sliceBitDepth+deltaX  (39)

It is noted that there is a relation ofchannelBitDepth≥sliceBitDepth≥deltaX. Representing the extension bitprecision dBD via the sequence bit depth and the slice bit depth makesit possible to make smaller the value of the difference parameterdeltaX. In other words, it is possible to decrease the code amountrequired for decoding/encoding the difference parameter deltaX.

When the processing of Step S234 is ended, then the dBD derivationprocessing is ended, and the processing returns to FIG. 26.

<Control Over Extension Bit Precision Control Group SG>

It is noted that the extension bit precision control group SG may becontrolled by a difference value diff_cu_delta_depth of a segmentationdepth with respect to a size Ctb Log 2Size of a CTB. In this case, asize Log 2SGSize of the SG is obtained as represented by the followingEquation (40).Log 2SGSize=Ctb Log 2Size−diff_cu_delta_depth  (40)

In a case of assuming, for example, that the CTB indicated by thedotted-line frame depicted in FIG. 20 has the size of 128×128 (Ctb Log2Size=7) and the SG indicated by the thick-line frame depicted in FIG.20 has the size of 32×32 (Log 2SGSize=5), the difference valuediff_cu_delta_depth is two as represented by the following Equation(41).diff_cu_delta_depth=Ctb Log 2Size−Log 2SGSize=7−5=2  (41)

It is noted that the difference value diff_cu_delta_depth is assumed tobe notified in units of predetermined parameter sets (headerinformation) (for example, a sequence parameter set (SPS), a pictureparameter set (PPS)) or by a slice header (SH) or the like). By doingso, it is possible to control granularity of the extension bit precisioncontrol group SG by the difference value diff_cu_delta_depth.

In a case, for example, in which a signal range is constant in a largearea to some extent, decreasing this difference valuediff_cu_delta_depth makes it possible to expand the unit of notificationof the difference parameter deltaX and to decrease overhead.

Conversely, in a case in which the signal range changes in a small area,increasing this difference value diff_cu_delta_depth makes it possibleto diminish the unit of notification of the difference parameter deltaXand to expand the number of significant figures of the transformcoefficient at the time of inverse quantization/inverse orthogonaltransform with finer granularity.

Other Example 1 of Definition of Extension Bit Precision dBD

While it is described above that the extension bit precision dBD of thecomponent X is obtained from the sequence bit depth channelBitDepth, theslice bit depth sliceBitDepth, and the extension bit precision dBD,definition of the extension bit precision dBD can be made freely and isnot limited to this example.

For example, the difference parameter deltaX may be defined as thedifference value between the sequence bit depth channelBitDepth and theextension bit precision dBD. In other words, the extension bit precisiondBD may be defined as the difference value between the sequence bitdepth channelBitDepth and the difference parameter deltaX, asrepresented by the following Equation (42)dBD=channelBitDepth−deltaX  (42)(that is, deltaX=channelBitDepth−dBD)

By doing so, it is possible to derive the extension bit precision dBDmore easily than the example described above.

Other Example 2 of Definition of Extension Bit Precision dBD

Moreover, the extension bit precision dBD may be defined to beequivalent to the difference parameter deltaX, as represented by thefollowing Equation (43).dBD=deltaX  (43)

In this case, the extension bit precision dBD can be derived withoutdependence on the parameters other than the difference parameter deltaX.

Other Example 3 of Definition of Extension Bit Precision dBD

Further, the extension bit precision dBD may be defined as a sum of thepredicted value dBD_(pred) of the extension bit precision dBD of theextension bit precision control group SG to be processed (referred to asa “current extension bit precision control group SG_(cur)”) and thedifference parameter deltaX, as represented by, for example, thefollowing Equation (44).dBD=dBD _(pred)+deltaX  (44)(that is, deltaX=dBD−dBD_(pred))

For example, this predicted value dBD_(pred) is derived by referring tothe encoded extension bit precision control groups neighborSGs in theneighborhood of the current extension bit precision control groupSG_(cur) as described above with reference to FIG. 21. Moreover, in thiscase, the predicted value dBD_(pred) of the extension bit precision dBDmay be set on the basis of the extension bit precision dBD_(A) of theneighboring extension bit precision control group SG_(A) and theextension bit precision dBD_(B) of the neighboring extension bitprecision control group SG₃. Furthermore, the predicted value dBD_(pred)of the extension bit precision dBD may be derived, for example, by amethod according to whether or not these neighboring extension bitprecision control groups can be referred to (for example, Equations (24)to (27) described above).

In such way, predicting the dBD of the current extension bit precisioncontrol group SG_(cur) by using each of the extension bit precision dBDof the encoded neighboring extension bit precision control groupsneighborSGs makes it possible to decrease the value of the differenceparameter deltaX. In other words, it is possible to suppress the growthof the code amount required for decoding or encoding the deltaX.

4. Second Embodiment

<Image Encoding Apparatus>

In the second embodiment, details of the method #2 in the table of FIG.9 will be described. First, configurations of exercising such controlover the computing precision at the time of encoding will be described.FIG. 28 is a block diagram depicting an example of principalconfigurations of the image encoding apparatus 100 in this case. Also inthe case of FIG. 28, the image encoding apparatus 100 basically hasconfigurations similar to those in the case of the method #1 (FIG. 15)except that the predicted image P obtained by the prediction section 122is also supplied to the control section 101.

For example, the control section 101 derives the extension bit precisiondBD per local level on the basis of this predicted image. Further, inthis case, the extension bit precision dBD is derived from the predictedimage; thus, the decoding side can similarly derive the extension bitprecision dBD from the predicted image. In other words, it isunnecessary to transmit the difference parameter deltaX. Therefore, thecontrol section 101 omits derivation of this difference parameterdeltaX. Naturally, the encoding section 115 omits encoding of thedifference parameter deltaX (information associated with the extensionbit precision).

<Details of Control Section>

FIG. 29 is a block diagram depicting an example of principalconfigurations of the control section 101 in this case, the principalconfigurations being related to generation of information associatedwith control over the number of significant figures. As depicted in FIG.29, the control section 101 in this case has the sequence bit depthsetting section 151, the pixel minimum value/maximum value searchsection 153, and the dBD derivation section 154. Since the transmissionof the difference parameter deltaX from the encoding side to thedecoding side is omitted, the deltaX derivation section 155 is omitted.Owing to this, the slice bit depth setting section 152 is also omitted.

Further, the predicted image is input to the pixel minimum value/maximumvalue search section 153, and the pixel minimum value/maximum valuesearch section 153 searches the minimum value (minPredPixelVal) and themaximum value (maxPredPixelVal) of pixel values of the predicted imageper local level that is a data unit smaller than the sequence level. Thepixel minimum value/maximum value search section 153 supplies theminimum value (minPredPixelVal) and the maximum value (maxPredPixelVal)of each local level detected by the search to the dBD derivation section154.

The sequence bit depth channelBitDepth supplied from the sequence bitdepth setting section 151 and the minimum value (minPredPixelVal) andthe maximum value (maxPredPixelVal) of the pixel values of the predictedimage supplied from the pixel minimum value/maximum value search section153 are input to the dBD derivation section 154, and the dBD derivationsection 154 derives the extension bit precision dBD per local level onthe basis of those parameters.

The dBD derivation section 154 supplies the derived extension bitprecision dBD to the orthogonal transform section 113, the quantizationsection 114, the inverse quantization section 117, the inverseorthogonal transform section 118, the expansion section 131, thenormalization section 132, the expansion section 133, and thenormalization section 134, as the transform information Tinfo.

In other words, the method #2 differs from the method #1 only in themethod of deriving the extension bit precision dBD and omission of thetransmission of the difference parameter deltaX and is basically similarto the method #1 in the other series of processing, for example, thecontrol over (expansion or normalization of) the number of significantfigures and the like in the expansion section 131, the normalizationsection 132, the expansion section 133, and the normalization section134.

In this case, therefore, it is possible to suppress a reduction inencoding efficiency, similarly to the method #1.

<Flow of Image Encoding Processing>

An example of a flow of the image encoding processing in this case willnext be described with reference to a flowchart of FIG. 30.

When the image encoding processing is started, processing of each ofSteps S301 and S302 is executed similarly to processing of each of StepsS101 and S102 of FIG. 17.

In Step S303, the control section 101 determines (sets) encodingparameters for each input image held by the reordering buffer 111. Inthis case, however, the control section 101 only sets the sequence bitdepth channelBitDepth on the basis of the input image and does notderive the extension bit precision dBD with respect to the informationassociated with the control over the number of significant figures.

In Step S304, the prediction section 122 performs prediction processingand generates the predicted image or the like in the optimum predictionmode, similarly to the case in Step S104.

In Step S305, the control section 101 derives the extension bitprecision dBD from the predicted image generated in Step S304.

Processing of each of Steps S306 to S319 is performed similarly to theprocessing of each of Steps S105 to S118 of FIG. 17.

When processing of Step S319 is ended, the image encoding processing isended.

In the image encoding processing in the flow described above, it ispossible to execute the orthogonal transform processing and thequantization processing with the improved computing precision, byexpanding the number of significant figures of the prediction residualresi that is yet to be subjected to orthogonal transform, with use ofthe extension bit precision dBD in Step S307, and by normalizing thequantized coefficient qcoefS subjected to quantization, with use of theextension bit precision dBD in Step S310. In other words, encodingefficiency is improved by the improved computing precision.

Likewise, it is possible to execute the inverse quantization processingand the inverse orthogonal transform processing with the improvedcomputing precision, by expanding the number of significant figures ofthe quantized coefficient qcoef that is yet to be subjected to inversequantization, with use of the extension bit precision dBD in Step S311,and by normalizing the prediction residual resiS′ subjected to inverseorthogonal transform, with use of the extension bit precision dBD inStep S314. In other words, encoding efficiency is improved by theimproved computing precision.

<Flow of Sequence Bit Depth Setting Processing>

In Step S303 of FIG. 30, the various encoding parameters are set. Forexample, the sequence bit depth channelBitDepth is set. An example of aflow of the sequence bit depth setting processing for setting thissequence bit depth channelBitDepth will be described with reference to aflowchart of FIG. 31.

When the sequence bit depth setting processing is started, the sequencebit depth setting section 151 obtains the bit depth of each component ofthe input image input from outside and sets the value of the obtainedbit depth to the sequence bit depth channelBitDepth of each component inStep S331.

When the processing of Step S331 is ended, then sequence bit depthsetting processing is ended, and the processing returns to FIG. 30.

<Flow of dBD Derivation Processing>

An example of a flow of the dBD derivation processing executed in theprocessing of Step S305 of FIG. 30 will be described with reference to aflowchart of FIG. 32.

When the dBD derivation processing is started, the pixel minimumvalue/maximum value search section 153 derives the minimum pixel valueminPredPixelVal and the maximum pixel value maxPredPixelVal of thecomponent X of the predicted image per extension bit precision controlgroup SG in Step S351.

In Step S352, the dBD derivation section 154 derives the extension bitprecision dBD of the component X per extension bit precision controlgroup SG, on the basis of the minimum pixel value minPredPixelVal andthe maximum pixel value maxPredPixelVal of the predicted image derivedin Step S351.

When the processing of Step S352 is ended, then the dBD derivationprocessing is ended, and the processing returns to FIG. 30.

As described above, the extension bit precision dBD is derived perextension bit precision control group SG; thus, the expansion section131, the normalization section 132, the expansion section 133, and thenormalization section 134 can each control the number of significantfigures of the coefficient (computing precision) per extension bitprecision control group SG by using the extension bit precision dBD, asin the case of the method #1. In this case, therefore, it is possible tosuppress a reduction in encoding efficiency, as in the case of themethod #1.

<Flow of Extension Bit Precision Information Encoding Processing>

In the processing of Step S318 of FIG. 30, the information associatedwith the sequence bit depth is encoded as the information associatedwith the control over the number of significant figures (control overthe computing precision) and is contained in the bit stream.

An example of a flow of extension bit precision information encodingprocessing for encoding this information associated with the sequencebit depth will be described with reference to a flowchart of FIG. 33.

When the extension bit precision information encoding processing isstarted, the encoding section 115 encodes the information associatedwith the sequence bit depth channelBitDepth of the component X andcontains the encoded information in the bit stream (for example,sequence parameter set SPS) in Step S371.

When processing of Step S371 is ended, then the extension bit precisioninformation encoding processing is ended, and the processing returns toFIG. 30.

As described above, in this case, the encoding section 115 encodes theinformation associated with the sequence bit depth and contains theencoded information in the bit stream (generates the bit streamcontaining the information associated with the sequence bit depth), butencoding of the difference parameter deltaX is omitted. It is thereforepossible to suppress a reduction in encoding efficiency.

Modification

In the image encoding apparatus according to the second embodiment(method #2 in the table of FIG. 9), the extension bit precision dBD perlocal level is derived by referring to the predicted image correspondingto the block to be processed. Alternatively, the image encodingapparatus may be configured such that a local bit depth of the block tobe processed is derived using the decoded image referred to at the timeof generation of the predicted image as an alternative to the predictedimage and that the extension bit precision dBD is derived by referringto the local bit depth (method #2′).

In other words, in FIG. 29, the decoded image Rec_(ref) referred to ingenerating the predicted image of the block to be processed is input tothe pixel minimum value/maximum value search section 153, and the pixelminimum value/maximum value search section 153 searches a minimum value(minRecPixelVal) and a maximum value (maxRecPixelVal) of pixel values ofthe decoded image Rec_(ref), per local level that is a data unit smallerthan the sequence level. The pixel minimum value/maximum value searchsection 153 supplies the minimum value (minRecPixelVal) and the maximumvalue (maxRecPixelVal) of each local level detected by the search to thedBD derivation section 154.

Further, the sequence bit depth channelBitDepth supplied from thesequence bit depth setting section 151 and the minimum value(minRecPixelVal) and the maximum value (maxRecPixelVal) of the pixelvalues of the decoded image Rec_(ref) supplied from the pixel minimumvalue/maximum value search section 153 are input to the dBD derivationsection 154, and the dBD derivation section 154 derives the extensionbit precision dBD per local level on the basis of those parameters.

Likewise, it is interpreted that in Step S305 of FIG. 30, ‘the extensionbit precision dBD is derived from the decoded image.’

In this case, therefore, it is possible to suppress a reduction inencoding efficiency, as in the case of the method #2. Furthermore, theadvantage of the method #2′ over the method #2 is that the extension bitprecision dBD can be derived only from the decoded image without theneed to generate the predicted image.

It is noted that the decoded image Rec_(ref) in the above descriptionindicates the local decoded pixel area referred to for generating anintra predicted image of a block to be processed in the case of intraprediction and indicates a local decoded pixel area referred to forgenerating an inter predicted image of a block to be processed in thecase of inter prediction.

<Image Decoding Apparatus>

Next, configurations of controlling the computing precision in themethod #2 in the table of FIG. 9 at the time of decoding will bedescribed. FIG. 34 is a block diagram depicting an example of principalconfigurations of the image decoding apparatus 200 in this case. Also inthe case of FIG. 34, the image decoding apparatus 200 basically hasconfigurations similar to those in the case of the method #1 (FIG. 24)except that the predicted image P obtained by the prediction section 219is also supplied to the decoding section 212.

For example, the decoding section 212 derives the extension bitprecision dBD per local level on the basis of this predicted image. Inother words, it is unnecessary to transmit the difference parameterdeltaX. It is therefore possible to suppress a reduction in encodingefficiency. In addition, the decoding section 212 naturally omitsdecoding of this difference parameter deltaX.

<Details of Decoding Section>

FIG. 35 is a block diagram depicting an example of principalconfigurations of the decoding section 212 in this case, the principalconfigurations being related to extraction of information associatedwith control over the number of significant figures. As depicted in FIG.35, the decoding section 212 in this case has the sequence bit depthderivation section 251, the dBD derivation section 254, and a pixelminimum value/maximum value search section 451. Since the transmissionof the difference parameter deltaX from the encoding side to thedecoding side is omitted, the slice bit depth derivation section 252 andthe deltaX decoding section 253 are omitted.

The predicted image is input to the pixel minimum value/maximum valuesearch section 451, and the pixel minimum value/maximum value searchsection 451 searches the minimum value (minPredPixelVal) and the maximumvalue (maxPredPixelVal) of pixel values of the predicted image, perlocal level that is a data unit smaller than the sequence level. Thepixel minimum value/maximum value search section 451 supplies theminimum value (minPredPixelVal) and the maximum value (maxPredPixelVal)of each local level detected by the search to the dBD derivation section254.

The sequence bit depth channelBitDepth supplied from the sequence bitdepth derivation section 251 and the minimum value (minPredPixelVal) andthe maximum value (maxPredPixelVal) of the pixel values of the predictedimage supplied from the pixel minimum value/maximum value search section451 are input to the dBD derivation section 254, and the dBD derivationsection 254 derives the extension bit precision dBD per local level onthe basis of those parameters.

The dBD derivation section 254 supplies the derived extension bitprecision dBD to the inverse quantization section 213, the inverseorthogonal transform section 214, the expansion section 231, and thenormalization section 232, as the transform information Tinfo.

In other words, the method #2 differs from the method #1 only in themethod of deriving the extension bit precision dBD and omission of thetransmission of the difference parameter deltaX in the decoding side aswell and is basically similar to the method #1 in the other series ofprocessing, for example, the control over the number of significantfigures and the like in the expansion section 231 and the normalizationsection 232.

In this case, therefore, it is possible to suppress a reduction inencoding efficiency, as in the case of the method #1.

<Flow of Image Decoding Processing>

An example of a flow of the image decoding processing in this case willnext be described with reference to a flowchart of FIG. 36.

When the image decoding processing is started, processing of Step S401is executed similarly to processing of Step S201 of FIG. 26.

In Step S402, the decoding section 212 performs decoding processing. Inthe case of this method #2, processing such as decoding of thedifference parameter deltaX and derivation of the slice bit depthsliceBitDepth is omitted (not performed) in this decoding processing.

In Step S403, the prediction section 219 performs prediction processingand generates the predicted image P, similarly in the case of Step S207.

In Step S404, the decoding section 212 derives the extension bitprecision dBD from the predicted image P generated in Step S403. It isnoted that this derivation of the extension bit precision dBD isperformed similarly to the encoding side (FIG. 32).

Processing of each of Steps S405 to S412 is performed similarly to theprocessing of each of Steps S203 to S206 and Steps S208 to S211. Inother words, the expansion section 231 and the normalization section 232exercise control over the number of significant figures (control overthe computing precision) with use of the extension bit precision dBD ofthe local level, similarly to the case of the method #1. In this case,therefore, it is similarly possible to suppress a reduction in encodingefficiency.

<Flow of Sequence Bit Depth Derivation Processing>

In the decoding processing of Step S402 of FIG. 36, the various encodingparameters are set. For example, the information associated with thesequence bit depth is read out from the bit stream, and the sequence bitdepth channelBitDepth is derived from the information associated withthe sequence bit depth. An example of a flow of the sequence bit depthderivation processing for deriving this sequence bit depthchannelBitDepth will be described with reference to a flowchart of FIG.36.

When the sequence bit depth derivation processing is started, thesequence bit depth derivation section 251 decodes the bit stream,extracts the information associated with the sequence bit depth, andderives the sequence bit depth channelBitDepth of each component on thebasis of the extracted information associated with the sequence bitdepth in Step S431.

When the processing of Step S431 is ended, then the sequence bit depthderivation processing is ended, and the processing returns to FIG. 36.

By executing each processing in such way, it is possible to exercisecontrol over the number of significant figures of the transformcoefficient per local level that is a data unit smaller than thesequence level. In the case of the method #2, therefore, it is similarlypossible to suppress a reduction in encoding efficiency.

Modification

In the image decoding apparatus according to the second embodiment(method #2 in the table of FIG. 9), the extension bit precision dBD perlocal level is derived by referring to the predicted image correspondingto the block to be processed. Alternatively, the image decodingapparatus may be configured such that the local bit depth of the blockto be processed is derived using the decoded image referred to at thetime of generation of the predicted image as an alternative to thepredicted image and that the extension bit precision dBD is derived byreferring to the local bit depth (method #2′).

In other words, in FIG. 35, the decoded image Rec_(ref) referred to ingenerating the predicted image of the block to be processed is input tothe pixel minimum value/maximum value search section 451, and the pixelminimum value/maximum value search section 451 searches the minimumvalue (minRecPixelVal) and the maximum value (maxRecPixelVal) of pixelvalues of the decoded image Rec_(ref), per local level that is a dataunit smaller than the sequence level. The pixel minimum value/maximumvalue search section 451 supplies the minimum value (minRecPixelVal) andthe maximum value (maxRecPixelVal) of each local level detected by thesearch to the dBD derivation section 254.

Further, the sequence bit depth channelBitDepth supplied from thesequence bit depth setting section 251 and the minimum value(minRecPixelVal) and the maximum value (maxRecPixelVal) of the pixelvalues of the decoded image Rec_(ref) supplied from the pixel minimumvalue/maximum value search section 451 are input to the dBD derivationsection 254, and the dBD derivation section 254 derives the extensionbit precision dBD per local level on the basis of those parameters.

Likewise, it is interpreted that in Step S404 of FIG. 36, ‘the extensionbit precision dBD is derived from the decoded image.’

In this case, therefore, it is possible to suppress a reduction inencoding efficiency, as in the case of the method #2. Furthermore, theadvantage of the method #2′ over the method #2 is that the extension bitprecision dBD can be derived only from the decoded image without theneed to generate the predicted image.

It is noted that the decoded image Rec_(ref) in the above descriptionindicates a local decoded pixel area referred to for generating an intrapredicted image of a block to be processed in a case of intra predictionand indicates a local decoded pixel area referred to for generating aninter predicted image of the block to be processed in a case of interprediction.

5. Notes

<Computer>

A series of processing described above can be executed either byhardware or by software. In a case of executing the series of processingby the software, a program configuring the software is installed into acomputer. Here, types of the computer include a computer incorporatedinto dedicated hardware and a computer, for example, a general-purposepersonal computer, capable of executing various functions by installingvarious programs into the computer.

FIG. 38 is a block diagram illustrating an example of a configuration ofthe hardware of the computer executing the series of processes describedabove by the program.

In a computer 800 depicted in FIG. 38, a CPU (Central Processing Unit)801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803are mutually connected by a bus 804.

An input/output interface 810 is also connected to the bus 804. An inputsection 811, an output section 812, a storage section 813, acommunication section 814, and a drive 815 are connected to theinput/output interface 810.

The input section 811 includes, for example, a keyboard, a mouse, amicrophone, a touch panel, and an input terminal. The output section 812includes, for example, a display, a speaker, and an output terminal. Thestorage section 813 includes, for example, a hard disk, a RAM disk, anda nonvolatile memory. The communication section 814 includes, forexample, a network interface. The drive 815 drives a removable medium821 such as a magnetic disk, an optical disk, a magneto-optical disk, ora semiconductor memory.

In the computer configured as described above, the CPU 801 loads aprogram stored in, for example, the storage section 813 to the RAM 803via the input/output interface 810 and the bus 804 and executes theprogram, whereby the series of processing described above is performed.Data and the like necessary for the CPU 801 to execute various kinds ofprocessing are also stored in the RAM 803 as appropriate.

The program to be executed by the computer (CPU 801) can be applied by,for example, recording the program in a removable medium 821 serving asa package medium or the like. In that case, the program can be installedinto the storage section 813 via the input/output interface 810 byattaching the removable medium 821 to the drive 815.

Alternatively, the program can be provided via a wired or wirelesstransmission medium such as a local area network, the Internet, or adigital satellite service. In that case, the program can be received bythe communication section 814 and installed into the storage section813.

As another alternative, this program can be installed into the ROM 802or the storage section 813 in advance.

<Unit of Information/Processing>

The data unit in which the various kinds of information described so farare set and the data unit to be subjected to various kinds of processingcan each be selected freely and are not limited to the examplesdescribed above. For example, the information or the processing may beset per TU (Transform Unit), TB (Transform Block), PU (Prediction Unit),PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit),sub-block, block, tile, slice, picture, sequence, or component, and datain the data unit may be subjected to the processing. Needless to say,this data unit can be set per information or per processing, and it isnot always necessary to use a uniform data unit in all information orall processing. It is noted that a storage location of these pieces ofinformation can be selected freely, and these pieces of information maybe stored in the header, the parameter set, or the like in the data unitdescribed above. Furthermore, these pieces of information may be storedin plural locations.

<Control Information>

Control information related to the present technology described in theembodiments so far may be transmitted from the encoding side to thedecoding side. For example, control information (for example,enabled_flag) for controlling whether or not to permit (or prohibit)application of the present technology described above may betransmitted. Alternatively, control information indicating, for example,an object to which the present technology described above is to beapplied (or an object to which the present technology is not to beapplied) may be transmitted. For example, control information fordesignating a block size (one of or both an upper limit and a lowerlimit of the block size), a frame, a component, a layer, or the like towhich the present technology is to be applied (or for permitting orprohibiting the application) may be transmitted.

<Objects to which Present Technology is Applied>

The present technology is applicable to any image encoding/decodingapproach. In other words, without contradiction with the presenttechnology described above, various kinds of processing associated withthe image encoding/decoding such as the transform (inverse transform),the quantization (inverse quantization), the encoding (decoding), andthe prediction may have freely-selected specifications, and thespecifications are not limited to the examples described above. Inaddition, without contradiction with the present technology describedabove, part of these kinds of processing may be omitted.

Moreover, the present technology is applicable to a multiview imageencoding/decoding system for encoding/decoding a multiview imagecontaining images at plural views. In that case, the present technologymay be applied to encoding/decoding of each view.

Furthermore, the present technology is applicable to a hierarchicalimage encoding (scalable encoding)/decoding system for encoding/decodinga hierarchical image that has plural layers (that are hierarchized) insuch a manner as to have scalability for predetermined parameters. Inthat case, the present technology may be applied to encoding/decoding ofeach hierarchy (layer).

The image encoding apparatus 100 and the image decoding apparatus 200according to the embodiments described above are applicable to variouselectronic apparatuses, such as a transmitter and a receiver (forexample, a television receiver and a cellular telephone) in distributionon satellite broadcasting, wired broadcasting for a cable TV and thelike, and the Internet and in distribution to a terminal by cellularcommunication and apparatuses (for example, a hard disk recorder and acamera) for recording images in a medium such as an optical disk, amagnetic disk, and a flash memory and reproducing images from thesestorage media.

Moreover, the present technology can be carried out as any configurationmounted in a freely selected apparatus or an apparatus configuring asystem, for example, as a processor (for example, video processor)serving as system LSI (Large Scale Integration), a module (for example,video module) using a plurality of processors or the like, a unit (forexample, video unit) using a plurality of modules or the like, a set(for example, video set) obtained by further adding other functions tothe unit, or the like (that is, as partial configurations of theapparatus).

Further, the present technology is also applicable to a network systemconfigured with plural apparatuses. For example, the present technologyis applicable to a cloud service for providing services associated withimages (image sequence) to any terminal such as an AV (Audio Visual)apparatus, a mobile information processing terminal, or an IoT (Internetof Things) device.

It is noted that systems, apparatuses, processing sections, and the liketo which the present technology is applied can be used in any field, forexample, a field of transportation, medicine, crime prevention,agriculture, livestock, mining, beauty, factories, consumer electronics,weather, and nature monitoring. In addition, use applications of thepresent technology may be determined freely.

For example, the present technology is applicable to a system or adevice used for providing listening and viewing content. In addition,the present technology is applicable to, for example, a system or adevice used for transportation such as monitoring of a traffic situationand autonomous driving control. Moreover, the present technology isapplicable to, for example, a system or a device used for security.Further, the present technology is applicable to, for example, a systemor a device used for automatic control over machines and the like.Further, the present technology is applicable to, for example, a systemor a device used for agriculture and livestock businesses. Further, thepresent technology is applicable to, for example, a system or a devicefor monitoring states of nature such as volcanos, forests, and oceans,wildlife, and the like. Furthermore, the present technology isapplicable to, for example, a system or a device used for sports.

<Others>

It is noted that in the present specification, a “flag” is informationfor identifying plural states and includes not only information for useat a time of identifying two states of true (1) and false (0) but alsoinformation by which three or more states can be identified. Therefore,a value which this “flag” possibly takes may be binary such as 1 or 0 ormay be three or more values. In other words, the number of bitsconfiguring this “flag” can be selected freely and may be one or may betwo or more. Moreover, not only a form of containing identificationinformation (including the flag) in a bit stream (a form of generating abit stream containing identification information) but also a form ofcontaining difference information regarding identification informationwith respect to information that forms a certain basis (a form ofgenerating a bit stream containing difference information regardingidentification information) is supposed; thus, in the presentspecification, the “flag” or the “identification information”encompasses not only the information but also the difference informationwith respect to the information that forms the basis.

Furthermore, various kinds of information (such as metadata) related toencoded data (bit stream) may be transmitted or recorded in any form aslong as the various kinds of information is associated with the encodeddata. A term “associate” means herein, for example, to allow the otherdata to be used (linked) at a time of processing one data. In otherwords, pieces of data associated with each other may be compiled as onedata or individual pieces of data. For example, information associatedwith the encoded data (image) may be transmitted on a transmission linedifferent from a transmission line used to transmit the encoded data(image). Moreover, the information associated with the encoded data(image) may be recorded, for example, in a recording medium differentfrom a recording medium in which the encoded data (image) is recorded(or in a different recording area in the same recording medium). It isnoted that this “association” may be association of part of data insteadof overall data. For example, an image and information corresponding tothe image may be associated with each other in any unit such as pluralframes, one frame, or a portion in a frame.

It is noted that in the present specification, terms such as “combine,”“multiplex,” “add,” “integrate,” “contain/include,” “store,”“incorporate,” “plug,” and “insert” mean to compile plural things intoone, for example, to compile the encoded data and the metadata into onedata, and mean one method for the “association” described above.

Moreover, the embodiments of the present technology are not limited tothe embodiments described above, and various changes can be made withoutdeparting from the spirit of the present technology.

Furthermore, the present technology can be carried out as anyconfiguration configuring an apparatus or a system, for example, as aprocessor as system LSI (Large Scale Integration), a module using aplurality of processors or the like, a unit using a plurality of modulesor the like, or a set obtained by further adding other functions to aunit (that is, configuration of part of an apparatus).

It is noted that a system means in the present specification acollection of plural constituent elements (apparatuses, modules (parts),and the like), regardless of whether all the constituent elements areprovided in the same casing. Therefore, plural apparatuses accommodatedin different casings and connected to one another via a network and oneapparatus in which plural modules are accommodated in one casing canboth be referred to as the “system.”

Further, the configurations described as one apparatus (or processingsection), for example, may be divided and configured as pluralapparatuses (or processing sections). Conversely, configurationsdescribed above as plural apparatuses (or processing sections) may becompiled and configured as one apparatus (or one processing section).Moreover, needless to say, configurations other than those of eachapparatus (or each processing section) described above may be added tothe configurations of each apparatus (or each processing section).Furthermore, if the configurations or operations are substantiallyidentical as an overall system, part of configurations of a certainapparatus (or a certain processing section) may be included in theconfigurations of another apparatus (or another processing section).

For example, the present technology can adopt a cloud computingconfiguration for causing plural apparatuses to process one function ina sharing or cooperative fashion.

Further, the program described above can be executed by, for example,any apparatus. In that case, the apparatus is only required to beconfigured with necessary functions (functional blocks or the like) tobe capable of obtaining necessary information.

Furthermore, each step described in the above flowcharts, for example,can be executed not only by one apparatus but also by plural apparatusesin a sharing fashion. Moreover, in a case in which one step includesplural series of processing, the plural series of processing included inthe one step can be executed not only by one apparatus but also byplural apparatuses in a sharing fashion. In other words, the pluralseries of processing included in the one step can be executed asprocessing of plural steps. Conversely, processing described as pluralsteps may be compiled into one step and executed collectively.

It is noted that the program to be executed by a computer may be aprogram for performing processing of steps describing the program intime series in an order described in the present specification or may bea program for individually performing the processing either in parallelor at necessary timing such as a timing of calling. In other words, theseries of processing in the steps may be executed in an order differentfrom the order described above unless contradiction arises. Furthermore,the processing in the steps that describe this program may be executedin parallel to processing of other programs or may be executed incombination with the processing of other programs.

The present technologies described in plural numbers in the presentspecification can be carried out independently and solely unlesscontradiction arises. Needless to say, the freely-selected pluralpresent technologies can be carried out in combination. For example,part of or entirety of the present technology described in any of theembodiments may be combined with part of or entirety of the presenttechnology described in another embodiment, and the combination can becarried out. Furthermore, part of or entirety of the freely-selectedpresent technology described above can be combined with anothertechnology that is not described above, and the combination of thetechnologies can be carried out.

It is noted that the present technology can also be configured asfollows.

(1)

An image processing apparatus including:

an expansion section that expands the number of significant figures of aprediction residual of an image on the basis of a bit depth indicating arange of pixel values of a local level that is a data unit smaller thana sequence level of the image;

a normalization section that normalizes the number of significantfigures of a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level; and

an encoding section that encodes the quantized coefficient the number ofsignificant figures of which is normalized by the normalization sectionand that generates a bit stream.

(2)

The image processing apparatus according to (1), in which

the expansion section bit-shifts the prediction residual by a shiftamount according to the bit depth of the local level, and

the normalization section bit-shifts the quantized coefficient by ashift amount according to the bit depth of the local level.

(3)

The image processing apparatus according to (1) or (2), furtherincluding:

an extension bit precision derivation section that derives extension bitprecision indicating a controlled variable for expansion of the numberof significant figures of the prediction residual and normalization ofthe quantized coefficient, on the basis of the bit depth of the locallevel, in which

the expansion section expands the number of significant figures of theprediction residual by the extension bit precision derived by theextension bit precision derivation section, and

the normalization section normalizes the number of significant figuresof the quantized coefficient by the extension bit precision derived bythe extension bit precision derivation section.

(4)

The image processing apparatus according to (3), in which

the extension bit precision derivation section derives a differencebetween the bit depth of the sequence level and the bit depth of thelocal level, as the extension bit precision.

(5)

The image processing apparatus according to (4), further including:

a sequence bit depth setting section that sets the bit depth of thesequence level on the basis of external parameters, in which

the extension bit precision derivation section derives the difference byusing the bit depth of the sequence level set by the sequence bit depthsetting section.

(6)

The image processing apparatus according to (4) or (5), furtherincluding:

a pixel minimum value/maximum value search section that searches aminimum value and a maximum value of the local level of pixel values ofthe image, in which

the extension bit precision derivation section

-   -   derives the bit depth of the local level by using a difference        between the maximum value and the minimum value detected by        search performed by the pixel minimum value/maximum value search        section, and    -   derives the difference by using the derived bit depth of the        local level.        (7)

The image processing apparatus according to any one of (3) to (6), inwhich

the extension bit precision derivation section derives the extension bitprecision per area at a size determined on the basis of a CTB size and adifference value of a segmentation depth with respect to the CTB size.

(8)

The image processing apparatus according to (7), in which

the encoding section encodes the difference value of the segmentationdepth with respect to the CTB size and generates the bit streamcontaining the difference value of the split depth with respect to theCTB size.

(9)

The image processing apparatus according to any one of (3) to (8), inwhich

the encoding section encodes information associated with the extensionbit precision derived by the extension bit precision derivation sectionand generates the bit stream containing the information associated withthe extension bit precision.

(10)

The image processing apparatus according to (9), further including:

a difference parameter derivation section that derives a differenceparameter by using the extension bit precision derived by the extensionbit precision derivation section, in which

the encoding section encodes the difference parameter derived by thedifference parameter derivation section, as the information associatedwith the extension bit precision, and generates the bit streamcontaining the difference parameter deltaX.

(11)

The image processing apparatus according to (10), in which

the difference parameter includes a difference between the bit depth ofa slice level of the image and the bit depth of the local level.

(12)

The image processing apparatus according to (11), in which

the encoding section encodes information associated with the bit depthof the sequence level and generates the bit stream containing theinformation associated with the bit depth of the sequence level.

(13)

The image processing apparatus according to (11) or (12), in which

the encoding section encodes information associated with the bit depthof the slice level and generates the bit stream containing theinformation associated with the bit depth of the slice level.

(14)

The image processing apparatus according to (13), further including:

a slice bit depth setting section that sets the bit depth of the slicelevel on the basis of a minimum value and a maximum value of the slicelevel of pixel values of the image, in which

the encoding section encodes the information associated with the bitdepth of the slice level set by the slice bit depth setting section andgenerates the bit stream containing the information associated with thebit depth of the slice level.

(15)

The image processing apparatus according to any one of (10) to (14), inwhich

the difference parameter includes a difference between the bit depth ofthe sequence level of the image and the bit depth of the local level.

(16)

The image processing apparatus according to any one of (10) to (15), inwhich

the difference parameter includes a difference between the extension bitprecision and a predicted value of the extension bit precision.

(17)

The image processing apparatus according to (16), in which

the predicted value of a current area corresponding to the local levelis derived on the basis of the extension bit precisions of neighboringareas of the current area corresponding to the local level.

(18)

The image processing apparatus according to any one of (1) to (17), inwhich

the encoding section encodes control information indicating whether thenumber of significant figures can be controlled on the basis of the bitdepth of the local level and generates the bit stream containing thecontrol information.

(19)

The image processing apparatus according to any one of (1) to (18),further including:

an orthogonal transform section that orthogonally transforms theprediction residual the number of significant figures of which isexpanded by the expansion section; and

a quantization section that quantizes coefficient data obtained byperformance of orthogonal transform on the prediction residual by theorthogonal transform section, in which

the normalization section normalizes the number of significant figuresof the quantized coefficient obtained by quantization of the coefficientdata by the quantization section.

(20)

An image processing method including;

expanding the number of significant figures of a prediction residual ofan image on the basis of a bit depth indicating a range of pixel valuesof a local level that is a data unit smaller than a sequence level ofthe image;

normalizing the number of significant figures of a quantized coefficientobtained by performance of orthogonal transform and quantization on theprediction residual the number of significant figures of which isexpanded, on the basis of the bit depth of the local level; and

encoding the quantized coefficient the number of significant figures ofwhich is normalized, and generating a bit stream.

(21)

An image processing apparatus including:

a decoding section that decodes a bit stream;

an expansion section that expands the number of significant figures of aquantized coefficient obtained by decoding of the bit stream by thedecoding section, on the basis of a bit depth indicating a range ofpixel values of a local level that is a data unit smaller than asequence level; and

a normalization section that normalizes the number of significantfigures of residual data obtained by performance of inverse quantizationand inverse orthogonal transform on the quantized coefficient the numberof significant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level.

(22)

The image processing apparatus according to (21), in which

the expansion section bit-shifts the quantized coefficient by a shiftamount according to the bit depth of the local level, and

the normalization section bit-shifts the residual data by a shift amountaccording to the bit depth of the local level.

(23)

The image processing apparatus according to (21) or (22), in which

the bit stream contains information associated with extension bitprecision indicating a controlled variable for expansion of the numberof significant figures of the quantized coefficient and normalization ofthe number of significant figures of the residual data,

the decoding section

-   -   decodes the bit stream and extracts the information associated        with the extension bit precision and contained in the bit        stream, and    -   derives the extension bit precision on the basis of the        extracted information associated with the extension bit        precision,

the expansion section expands the number of significant figures of thequantized coefficient by the extension bit precision derived by thedecoding section, and

the normalization section normalizes the number of significant figuresof the residual data by the extension bit precision derived by thedecoding section.

(24)

The image processing apparatus according to (23), in which

the extension bit precision includes a difference between the bit depthof the sequence level and the bit depth of the local level.

(25)

The image processing apparatus according to (24), in which

the information associated with the extension bit precision contains adifference parameter that includes a difference between the bit depth ofa slice level of the image and the bit depth of the local level, and

the decoding section derives the extension bit precision by subtractingthe bit depth of the slice level from the bit depth of the sequencelevel and adding the difference parameter to a subtraction result.

(26)

The image processing apparatus according to (25), in which

the bit stream further contains information associated with the bitdepth of the sequence level, and

the decoding section

-   -   decodes the bit stream and extracts the information associated        with the bit depth of the sequence level and contained in the        bit stream,    -   derives the bit depth of the sequence level on the basis of the        extracted information associated with the bit depth of the        sequence level, and    -   derives the extension bit precision by using the derived bit        depth of the sequence level.        (27)

The image processing apparatus according to (26), in which

the information associated with the bit depth of the sequence levelincludes a value obtained by subtracting a predetermined value from thebit depth of the sequence level, and

the decoding section derives the bit depth of the sequence level byadding the predetermined value to the information associated with thebit depth of the sequence level.

(28)

The image processing apparatus according to any one of (25) to (27), inwhich

the bit stream further contains information associated with the bitdepth of the slice level, and

the decoding section

-   -   decodes the bit stream and extracts the information associated        with the bit depth of the slice level and contained in the bit        stream,    -   derives the bit depth of the slice level on the basis of the        extracted information associated with the bit depth of the slice        level, and    -   derives the extension bit precision using the derived bit depth        of the slice level.        (29)

The image processing apparatus according to (28), in which

the information associated with the bit depth of the slice level is aminimum value and a maximum value of the slice level of pixel values ofthe image, and

the decoding section derives the bit depth of the slice level by using adifference between the maximum value and the minimum value.

(30)

The image processing apparatus according to any one of (24) to (29), inwhich

the information associated with the extension bit precision contains adifference parameter that includes a difference between the bit depth ofthe sequence level of the image and the bit depth of the local level,and

the decoding section derives the extension bit precision by subtractingthe difference parameter from the bit depth of the sequence level.

(31)

The image processing apparatus according to (30), in which

the bit stream further contains information associated with the bitdepth of the sequence level, and

the decoding section

-   -   decodes the bit stream and extracts the information associated        with the bit depth of the sequence level and contained in the        bit stream,    -   derives the bit depth of the sequence level on the basis of the        extracted information associated with the bit depth of the        sequence level, and    -   derives the extension bit precision using the derived bit depth        of the sequence level.        (32)

The image processing apparatus according to (31), in which

the information associated with the bit depth of the sequence levelincludes a value obtained by subtracting a predetermined value from thebit depth of the sequence level, and

the decoding section derives the bit depth of the sequence level byadding the predetermined value to the information associated with thebit depth of the sequence level.

(33)

The image processing apparatus according to any one of (24) to (32), inwhich

the information associated with the extension bit precision contains adifference parameter that includes a difference between the extensionbit precision and a predicted value of the extension bit precision, and

the decoding section derives the extension bit precision by adding thepredicted value to the difference parameter.

(34)

The image processing apparatus according to (33), in which

the predicted value of a current area corresponding to the local levelis derived on the basis of the extension bit precisions of neighboringareas of the current area corresponding to the local level.

(35)

The image processing apparatus according to (34), in which

the predicted value of the current area is derived by a method dependingon whether or not each of an area in the neighborhood of an upper sideof the current area and an area in the neighborhood of a left side ofthe current area can be referred to.

(36)

The image processing apparatus according to any one of (23) to (35), inwhich

the decoding section derives the extension bit precision per area at asize determined on the basis of a CTB size and a difference value of asegmentation depth with respect to the CTB size.

(37)

The image processing apparatus according to (36), in which

the decoding section decodes the bit stream and extracts the differencevalue of the segmentation depth with respect to the CTB size.

(38)

The image processing apparatus according to any one of (21) to (37), inwhich

the decoding section decodes the bit stream, extracts controlinformation indicating whether control over the number of significantfigures based on the bit depth of the local level can be exercised, andcontrols application of the control over the number of significantfigures based on the bit depth of the local level, according to theextracted control information.

(39)

The image processing apparatus according to any one of (21) to (38),further including:

an inverse quantization section that performs inverse quantization onthe quantized coefficient the number of significant figures of which isexpanded by the expansion section; and

an inverse orthogonal transform section that performs inverse orthogonaltransform on coefficient data obtained by performance of the inversequantization on the quantized coefficient by the inverse quantizationsection, in which

the normalization section normalizes the residual data obtained byperformance of the inverse orthogonal transform on the coefficient databy the inverse orthogonal transform section.

(40)

An image processing method including:

decoding a bit stream;

expanding the number of significant figures of a quantized coefficientobtained by decoding of the bit stream, on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level; and

normalizing the number of significant figures of residual data obtainedby performance of inverse quantization and inverse orthogonal transformon the quantized coefficient the number of significant figures of whichis expanded, on the basis of the bit depth of the local level.

(41)

An image processing apparatus including:

an expansion section that expands the number of significant figures of aprediction residual of an image on the basis of a bit depth indicating arange of pixel values of a local level that is a data unit smaller thana sequence level of a predicted image corresponding to the image or of adecoded image referred to at a time of generation of the predictedimage;

a normalization section that normalizes the number of significantfigures of a quantized coefficient obtained by performance of orthogonaltransform and quantization on the prediction residual the number ofsignificant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level of the predicted image orthe decoded image; and

an encoding section that encodes the quantized coefficient the number ofsignificant figures of which is normalized by the normalization sectionand that generates a bit stream.

(42)

The image processing apparatus according to (41), in which

the expansion section bit-shifts the prediction residual by a shiftamount according to the bit depth of the local level, and

the normalization section bit-shifts the quantized coefficient by ashift amount according to the bit depth of the local level.

(43)

The image processing apparatus according to (41) or (42), furtherincluding:

an extension bit precision derivation section that derives extension bitprecision indicating a controlled variable for expansion of the numberof significant figures of the prediction residual and normalization ofthe quantized coefficient, on the basis of the bit depth of the locallevel, in which

the expansion section expands the number of significant figures of theprediction residual by the extension bit precision derived by theextension bit precision derivation section, and

the normalization section normalizes the number of significant figuresof the quantized coefficient by the extension bit precision derived bythe extension bit precision derivation section.

(44)

The image processing apparatus according to (43), in which

the extension bit precision derivation section derives a differencebetween the bit depth of the sequence level and the bit depth of thelocal level of the predicted image, as the extension bit precision.

(45)

The image processing apparatus according to (44), further including:

a sequence bit depth setting section that sets the bit depth of thesequence level on the basis of external parameters, in which

the extension bit precision derivation section derives the difference byusing the bit depth of the sequence level set by the sequence bit depthsetting section.

(46)

The image processing apparatus according to (44) or (45), furtherincluding:

a prediction section that predicts the image and that generates thepredicted image, in which

the extension bit precision derivation section derives the difference byusing the bit depth of the local level of the predicted image generatedby the prediction section.

(47)

The image processing apparatus according to (46), further including:

a pixel minimum value/maximum value search section that searches aminimum value and a maximum value of the local level of pixel values ofthe predicted image generated by the prediction section, in which

the extension bit precision derivation section

-   -   derives the bit depth of the local level of the predicted image        by using a difference between the maximum value and the minimum        value detected by search performed by the pixel minimum        value/maximum value search section, and    -   derives the difference by using the derived bit depth of the        local level of the predicted image.        (48)

The image processing apparatus according to any one of (43) to (47), inwhich

the extension bit precision derivation section derives the extension bitprecision per area at a size determined on the basis of a CTB size and adifference value of a segmentation depth with respect to the CTB size.

(49)

The image processing apparatus according to (48), in which

the encoding section encodes the difference value of the segmentationdepth with respect to the CTB size and generates the bit streamcontaining the difference value of the split depth with respect to theCTB size.

(50)

The image processing apparatus according to any one of (41) to (49), inwhich

the encoding section encodes control information indicating whether thenumber of significant figures can be controlled on the basis of the bitdepth of the local level of the predicted image and generates the bitstream containing the control information.

(51)

The image processing apparatus according to any one of (41) to (50),further including:

an orthogonal transform section that orthogonally transforms theprediction residual the number of significant figures of which isexpanded by the expansion section; and

a quantization section that quantizes coefficient data obtained byperformance of orthogonal transform on the prediction residual by theorthogonal transform section, in which

the normalization section normalizes the number of significant figuresof the quantized coefficient obtained by quantization of the coefficientdata by the quantization section.

(52)

An image processing method including:

expanding the number of significant figures of a prediction residual ofan image on the basis of a bit depth indicating a range of pixel valuesof a local level that is a data unit smaller than a sequence level of apredicted image corresponding to the image or of a decoded imagereferred to at a time of generation of the predicted image;

normalizing the number of significant figures of a quantized coefficientobtained by performance of orthogonal transform and quantization on theprediction residual the number of significant figures of which isexpanded, on the basis of the bit depth of the local level of thepredicted image or the decoded image; and

encoding the quantized coefficient the number of significant figures ofwhich is normalized and generating a bit stream.

(61)

An image processing apparatus including:

a decoding section that decodes a bit stream;

an expansion section that expands the number of significant figures of aquantized coefficient obtained by decoding of the bit stream by thedecoding section, on the basis of a bit depth indicating a range ofpixel values of a local level that is a data unit smaller than asequence level of a predicted image or of a decoded image referred to ata time of generation of the predicted image; and

a normalization section that normalizes the number of significantfigures of residual data obtained by performance of inverse quantizationand inverse orthogonal transform on the quantized coefficient the numberof significant figures of which is expanded by the expansion section, onthe basis of the bit depth of the local level of the predicted image orthe decoded image.

(62)

The image processing apparatus according to (61), in which

the expansion section bit-shifts the quantized coefficient by a shiftamount according to the bit depth of the local level of the predictedimage, and

the normalization section bit-shifts the residual data by a shift amountaccording to the bit depth of the local level of the predicted image.

(63)

The image processing apparatus according to (61) or (62), furtherincluding:

an extension bit precision derivation section that derives extension bitprecision that includes a correction value of a controlled variable ofthe number of significant figures of the quantized coefficient and of acontrolled variable of the number of significant figures of the residualdata set per sequence of the image, on the basis of the bit depth of thelocal level of the predicted image, in which

the expansion section expands the number of significant figures of thequantized coefficient by the extension bit precision derived by theextension bit precision derivation section, and

the normalization section normalizes the number of significant figuresof the residual data by the extension bit precision derived by theextension bit precision derivation section.

(64)

The image processing apparatus according to (63), in which

the extension bit precision derivation section derives a differencebetween the bit depth of the sequence level and the bit depth of thelocal level of the predicted image, as the extension bit precision.

(65)

The image processing apparatus according to (64), further including:

a sequence bit depth setting section that sets the bit depth of thesequence level on the basis of external parameters, in which

the extension bit precision derivation section derives the difference byusing the bit depth of the sequence level set by the sequence bit depthsetting section.

(66)

The image processing apparatus according to (64) or (65), furtherincluding:

a prediction section that predicts the image and that generates thepredicted image, in which

the extension bit precision derivation section

-   -   derives the difference by using the bit depth of the local level        of the predicted image generated by the prediction section.        (67)

The image processing apparatus according to (66), further including:

a pixel minimum value/maximum value search section that searches aminimum value and a maximum value of the local level of pixel values ofthe predicted image generated by the prediction section, in which

the extension bit precision derivation section

-   -   derives the bit depth of the local level of the predicted image        by using a difference between the maximum value and the minimum        value detected by search performed by the pixel minimum        value/maximum value search section, and    -   derives the difference using the derived bit depth of the local        level of the predicted image.        (68)

The image processing apparatus according to any one of (63) to (67), inwhich

the extension bit precision derivation section derives the extension bitprecision per area at a size determined on the basis of a CTB size and adifference value of a segmentation depth with respect to the CTB size.

(69)

The image processing apparatus according to (68), in which

the decoding section decodes the bit stream and extracts the differencevalue of the segmentation depth with respect to the CTB size, and

the extension bit precision derivation section derives the extension bitprecision per area at the size determined on the basis of the CTB sizeand the difference value of the segmentation depth with respect to theCTB size.

(70)

The image processing apparatus according to any one of (61) to (69), inwhich

the decoding section decodes the bit stream, extracts controlinformation indicating whether application of expansion of the number ofsignificant figures based on the bit depth of the local level of thepredicted image is possible, and controls the application of theexpansion of the number of significant figures based on the bit depth ofthe local level of the predicted image, according to the extractedcontrol information.

(71)

The image processing apparatus according to any one of (61) to (70),further including:

an inverse quantization section that performs inverse quantization onthe quantized coefficient the number of significant figures of which isexpanded by the expansion section; and

an inverse orthogonal transform section that performs inverse orthogonaltransform on coefficient data obtained by performance of the inversequantization on the quantized coefficient by the inverse quantizationsection, in which

the normalization section normalizes the number of significant figuresof the residual data obtained by performance of the inverse orthogonaltransform on the coefficient data by the inverse orthogonal transformsection.

(72)

An image processing method including:

decoding a bit stream;

expanding the number of significant figures of a quantized coefficientobtained by decoding of the bit stream, on the basis of a bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of a predicted image or of a decoded imagereferred to at a time of generation of the predicted image; and

normalizing the number of significant figures of residual data obtainedby performance of inverse quantization and inverse orthogonal transformon the quantized coefficient the number of significant figures of whichis expanded, on the basis of the bit depth of the local level of thepredicted image or the decoded image.

REFERENCE SIGNS LIST

100: Image encoding apparatus, 101: Control section, 112: Computingsection, 113: Orthogonal transform section, 114: Quantization section,115: Encoding section, 117: Inverse quantization section, 118: Inverseorthogonal transform section, 122: Prediction section, 131: Expansionsection, 132: Normalization section, 133: Expansion section, 134:Normalization section, 151: Sequence bit depth derivation section, 152:Slice bit depth derivation section, 153: Pixel minimum value/maximumvalue search section, 154: dBD derivation section, 155: deltaXderivation section, 200: Image decoding apparatus, 212: Decodingsection, 213: Inverse quantization section, 214: Inverse orthogonaltransform section, 215: Computing section, 219: Prediction section, 231:Expansion section, 232: Normalization section, 251: Sequence bit depthderivation section, 252: Slice bit depth derivation section, 253: deltaXdecoding section, 254: dBD derivation section, 451: Pixel minimumvalue/maximum value search section

The invention claimed is:
 1. An image processing apparatus comprising:an expansion section configured to expand a number of significantfigures of a prediction residual of an image on a basis of a change of abit depth between different frames of the image in a time direction orbetween different areas of the image in a spatial direction, the bitdepth indicating a range of pixel values of a local level that is a dataunit smaller than a sequence level of the image; a normalization sectionconfigured to normalize the number of significant figures of a quantizedcoefficient obtained by performance of orthogonal transform andquantization on the prediction residual the number of significantfigures of which is expanded by the expansion section, on a basis of thechange of the bit depth of the local level; and an encoding sectionconfigured to encode the quantized coefficient the number of significantfigures of which is normalized by the normalization section and generatea bit stream, wherein the expansion section, the normalization section,and the encoding section are each implemented via at least oneprocessor.
 2. The image processing apparatus according to claim 1,wherein the expansion section is further configured to bit-shift theprediction residual by a shift amount according to the bit depth of thelocal level, and the normalization section is further configured tobit-shift the quantized coefficient by a shift amount according to thebit depth of the local level.
 3. The image processing apparatusaccording to claim 1, further comprising: an extension bit precisionderivation section configured to derive extension bit precisionindicating a controlled variable for expansion of the number ofsignificant figures of the prediction residual and normalization of thequantized coefficient, on a basis of the bit depth of the local level,wherein the expansion section is further configured to expand the numberof significant figures of the prediction residual by the extension bitprecision derived by the extension bit precision derivation section, thenormalization section is further configured to normalize the number ofsignificant figures of the quantized coefficient by the extension bitprecision derived by the extension bit precision derivation section, andthe extension bit precision derivation section is implemented via atleast one processor.
 4. The image processing apparatus according toclaim 3, wherein the extension bit precision derivation section isfurther configured to derive a difference between the bit depth of thesequence level and the bit depth of the local level, as the extensionbit precision.
 5. The image processing apparatus according to claim 3,wherein the encoding section is further configured to encode informationassociated with the extension bit precision derived by the extension bitprecision derivation section, and generate the bit stream containing theinformation associated with the extension bit precision.
 6. The imageprocessing apparatus according to claim 5, further comprising: adifference parameter derivation section configured to derive adifference parameter by using the extension bit precision derived by theextension bit precision derivation section, wherein the encoding sectionis further configured to encode the difference parameter derived by thedifference parameter derivation section, as the information associatedwith the extension bit precision, and generates the bit streamcontaining the difference parameter, and the difference parameterderivation section is implemented via at least one processor.
 7. Theimage processing apparatus according to claim 6, wherein the differenceparameter includes a difference between the bit depth of a slice levelof the image and the bit depth of the local level.
 8. An imageprocessing method comprising: expanding a number of significant figuresof a prediction residual of an image on a basis of a change of a bitdepth between different frames of the image in a time direction orbetween different areas of the image in a spatial direction, the bitdepth indicating a range of pixel values of a local level that is a dataunit smaller than a sequence level of the image; normalizing the numberof significant figures of a quantized coefficient obtained byperformance of orthogonal transform and quantization on the predictionresidual the number of significant figures of which is expanded, on abasis of the change of the bit depth of the local level; and encodingthe quantized coefficient the number of significant figures of which isnormalized, and generating a bit stream.
 9. An image processingapparatus comprising: an expansion section configured to expand a numberof significant figures of a prediction residual of an image on a basisof a change of a bit depth between different frames of the image in atime direction or between different areas of the image in a spatialdirection, the bit depth indicating a range of pixel values of a locallevel that is a data unit smaller than a sequence level of a predictedimage corresponding to the image or of a decoded image referred to at atime of generation of the predicted image; a normalization sectionconfigured to normalize the number of significant figures of a quantizedcoefficient obtained by performance of orthogonal transform andquantization on the prediction residual the number of significantfigures of which is expanded by the expansion section, on a basis of thechange of the bit depth of the local level of the predicted image or thedecoded image; and an encoding section configured to encode thequantized coefficient the number of significant figures of which isnormalized by the normalization section and generate a bit stream,wherein the expansion section, the normalization section, and theencoding section are each implemented via at least one processor.
 10. Animage processing method comprising: expanding a number of significantfigures of a prediction residual of an image on a basis of a change of abit depth between different frames of the image in a time direction orbetween different areas of the image in a spatial direction, the bitdepth indicating a range of pixel values of a local level that is a dataunit smaller than a sequence level of a predicted image corresponding tothe image or of a decoded image referred to at a time of generation ofthe predicted image; normalizing the number of significant figures of aquantized coefficient obtained by performance of orthogonal transformand quantization on the prediction residual the number of significantfigures of which is expanded, on a basis of the change of the bit depthof the local level of the predicted image or the decoded image; andencoding the quantized coefficient the number of significant figures ofwhich is normalized, and generating a bit stream.
 11. An imageprocessing apparatus comprising: a decoding section configured to decodea bit stream; an expansion section configured to expand a number ofsignificant figures of a quantized coefficient obtained by decoding ofthe bit stream by the decoding section, on a basis of a change of a bitdepth between different frames of the bit stream in a time direction orbetween different areas of the bit stream in a spatial direction, thebit depth indicating a range of pixel values of a local level that is adata unit smaller than a sequence level; and a normalization sectionconfigured to normalize a number of significant figures of residual dataobtained by performance of inverse quantization and inverse orthogonaltransform on the quantized coefficient the number of significant figuresof which is expanded by the expansion section, on a basis of the changeof the bit depth of the local level, wherein the decoding section, theexpansion section, and the normalization section are each implementedvia at least one processor.
 12. The image processing apparatus accordingto claim 11, wherein the expansion section is further configured tobit-shift the quantized coefficient by a shift amount according to thebit depth of the local level, and the normalization section is furtherconfigured to bit-shift the residual data by a shift amount according tothe bit depth of the local level.
 13. The image processing apparatusaccording to claim 11, wherein the bit stream contains informationassociated with extension bit precision indicating a controlled variablefor expansion of the number of significant figures of the quantizedcoefficient and normalization of the number of significant figures ofthe residual data, the decoding section is further configured to decodethe bit stream, and extract the information associated with theextension bit precision and contained in the bit stream, and derive theextension bit precision on a basis of the extracted informationassociated with the extension bit precision, the expansion section isfurther configured to expand the number of significant figures of thequantized coefficient by the extension bit precision derived by thedecoding section, and the normalization section is further configured tonormalize the number of significant figures of the residual data by theextension bit precision derived by the decoding section.
 14. The imageprocessing apparatus according to claim 13, wherein the extension bitprecision includes a difference between the bit depth of the sequencelevel and the bit depth of the local level.
 15. The image processingapparatus according to claim 13, wherein the decoding section is furtherconfigured to derive the extension bit precision per area at a sizedetermined on a basis of a Coded Tree Block (CTB) size and a differencevalue of a segmentation depth with respect to the CTB size.
 16. Theimage processing apparatus according to claim 15, wherein the decodingsection is further configured to decode the bit stream and extract thedifference value of the segmentation depth with respect to the CTB size.17. The image processing apparatus according to claim 11, wherein thedecoding section is further configured to decode the bit stream, extractcontrol information indicating whether control over the number ofsignificant figures based on the bit depth of the local level can beexercised, and control application of the control over the number ofsignificant figures based on the bit depth of the local level, accordingto the extracted control information.
 18. An image processing methodcomprising: decoding a bit stream; expanding a number of significantfigures of a quantized coefficient obtained by decoding of the bitstream, on a basis of a change of a bit depth between different framesof the bit stream in a time direction or between different areas of thebit stream in a spatial direction, the bit depth indicating a range ofpixel values of a local level that is a data unit smaller than asequence level; and normalizing a number of significant figures ofresidual data obtained by performance of inverse quantization andinverse orthogonal transform on the quantized coefficient the number ofsignificant figures of which is expanded, on a basis of the change ofthe bit depth of the local level.
 19. An image processing apparatuscomprising: a decoding section configured to decode a bit stream; anexpansion section configured to expand a number of significant figuresof a quantized coefficient obtained by decoding of the bit stream by thedecoding section, on a basis of a change of a bit depth betweendifferent frames of the bit stream in a time direction or betweendifferent areas of the bit stream in a spatial direction, the bit depthindicating a range of pixel values of a local level that is a data unitsmaller than a sequence level of a predicted image or of a decoded imagereferred to at a time of generation of the predicted image; and anormalization section configured to normalize a number of significantfigures of residual data obtained by performance of inverse quantizationand inverse orthogonal transform on the quantized coefficient the numberof significant figures of which is expanded by the expansion section, ona basis of the change of the bit depth of the local level, wherein thedecoding section, the expansion section, and the normalization sectionare each implemented via at least one processor.
 20. An image processingmethod comprising: decoding a bit stream; expanding a number ofsignificant figures of a quantized coefficient obtained by decoding ofthe bit stream, on a basis of a change of a bit depth between differentframes of the bit stream in a time direction or between different areasof the bit stream in a spatial direction, the bit depth indicating arange of pixel values of a local level that is a data unit smaller thana sequence level of a predicted image or of a decoded image referred toat a time of generation of the predicted image; and normalizing a numberof significant figures of residual data obtained by performance ofinverse quantization and inverse orthogonal transform on the quantizedcoefficient the number of significant figures of which is expanded, on abasis of the change of the bit depth of the local level of the predictedimage or the decoded image.
 21. The image processing apparatus accordingto claim 1, wherein the change in the bit depth between different framesof the image in the time direction includes a change between a bit depthof a first frame of the image and a bit depth of a second frame of theimage different than the first frame, and wherein the change in the bitdepth between different areas of the image in the spatial directionincludes a change between a bit depth of a first area of the image and abit depth of a second area of the image different than the first area.